-- Logs begin at Wed 2022-08-24 11:53:48 UTC, end at Wed 2022-08-24 12:52:45 UTC. -- Aug 24 12:18:30.386234 dv-cinder-846054-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 12:18:30.393833 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 12:18:30 2022] *** Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: nodename: dv-cinder-846054-1-iscsi Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: machine: x86_64 Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: clock source: unix Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: pcre jit disabled Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: detected number of CPU cores: 4 Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: current working directory: / Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: detected binary path: /usr/bin/uwsgi-core Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: your processes number limit is 47680 Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: your memory page size is 4096 bytes Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: detected max file descriptor number: 2048 Aug 24 12:18:30.395896 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: lock engine: pthread robust mutexes Aug 24 12:18:30.396817 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: thunder lock: enabled Aug 24 12:18:30.396817 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 12:18:30.396817 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 12:18:30.436853 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: Python main interpreter initialized at 0x561cd7a1b9c0 Aug 24 12:18:30.436853 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: python threads support enabled Aug 24 12:18:30.436853 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: your server socket listen backlog is limited to 100 connections Aug 24 12:18:30.436853 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: your mercy for graceful operations on workers is 90 seconds Aug 24 12:18:30.437239 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 12:18:30.437317 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: *** Operational MODE: preforking *** Aug 24 12:18:30.437579 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: *** uWSGI is running in multiple interpreter mode *** Aug 24 12:18:30.437579 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: spawned uWSGI master process (pid: 81622) Aug 24 12:18:30.437797 dv-cinder-846054-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 12:18:30.438849 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: spawned uWSGI worker 1 (pid: 81634, cores: 1) Aug 24 12:18:30.438849 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: spawned uWSGI worker 2 (pid: 81635, cores: 1) Aug 24 12:18:30.438849 dv-cinder-846054-1-iscsi devstack@keystone.service[81622]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 12:18:34.673305 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 12:18:34.673305 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: from cryptography.utils import int_from_bytes Aug 24 12:18:34.674142 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 12:18:34.674142 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: from cryptography.utils import int_from_bytes Aug 24 12:18:34.674283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 12:18:34.674283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: from cryptography.utils import int_from_bytes Aug 24 12:18:34.674283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 12:18:34.674283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: from cryptography.utils import int_from_bytes Aug 24 12:18:34.723862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 12:18:34.724044 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 12:18:34.724228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] command line args: [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 12:18:34.724370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 12:18:34.724450 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 12:18:34.724527 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 12:18:34.724705 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ================================================================================ {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 12:18:34.724825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] command line args: [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 12:18:34.724894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] admin_token = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.725031 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 12:18:34.725161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] config_dir = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.725234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ================================================================================ {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 12:18:34.725310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.725501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] admin_token = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.725631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] config_source = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.725631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] debug = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.725757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] config_dir = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.725815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.726143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.726143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] config_source = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.726257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] debug = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.726312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.726375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.726431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.726600 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.726679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.726743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.726849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.727037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.727119 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.727222 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_config_append = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.727293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.727389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.727465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.727564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_dir = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.727639 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] log_config_append = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.727736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_file = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.727811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.727943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] log_dir = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.728037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.728107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] log_file = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.728244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.728335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.728465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.728614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.728791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.728938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.729056 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.729132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.729323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.729564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.729663 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.729909 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.730003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.730163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.730415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.730510 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.730630 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.730846 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.730926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.731108 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.731199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.731343 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.731595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.731674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.731844 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.732052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.732138 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.732331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.732607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.732725 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.732842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] publish_errors = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.733046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.733220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.733302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.733415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.733556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] publish_errors = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.733648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.733867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.733947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.734092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.734230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] standard_threads = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.734312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.734390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.734520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.734599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.734726 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.734847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.734990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] use_journal = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.735067 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] use_json = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.735142 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] standard_threads = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.735219 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] use_stderr = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.735358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.735504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] use_syslog = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.735608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.735802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.735879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.736090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.736171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.736351 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.736430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] use_journal = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.736595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.736783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] use_json = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.736868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.737052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] use_stderr = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.737141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.737241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] use_syslog = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.737318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.737491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:34.737567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.737763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.737852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.738012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.738095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.738261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.738343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.738498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.738583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.738749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.738828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.738988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.739067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.739236 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.739324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.739475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.739552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.739712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.739789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.739966 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.740054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.740216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.740300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.740451 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.740527 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.740710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.740803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.740953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.741036 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.741200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.741278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.741449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.741604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.741678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.741747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.741916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.741994 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.742149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.742227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.742380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.742463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.742617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.742694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.742868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.742953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.743119 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.743202 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.743351 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.743428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.743585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.743663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.743823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.743899 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.744056 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.744237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.744333 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.744565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.744734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.744812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.744882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.744951 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.745083 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.745160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.745318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.745398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.745552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.745628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.745785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.745869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.746045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.746126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.746280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.746357 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.746520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.746598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.746771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.746849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.747005 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.747244 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.747398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.747472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.747547 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.747702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.747778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.747938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.748060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.748187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.748265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.748420 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.748502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.748685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.748779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.748975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.749055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.749124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.749192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.749367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.749481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.749585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.749680 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.749822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.749926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.750060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.750367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.750467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.750539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.750695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.750780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.750860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.750963 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.751042 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.751229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.751311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.751463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.751542 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.751694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.751773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.751921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.751997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.752159 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.752247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.752392 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.752521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.752630 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.752771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.752890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.752979 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.753150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.753236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.753387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.753465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.753620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.753701 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.753854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.753983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.754087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.754164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.754342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.754421 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.754583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.754670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.754828 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.754905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.755069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.755146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.755307 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.755397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.755539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.755616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.755776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.755853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.756015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.756103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.756251 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.756334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.756489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.756568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.756756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.756851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.756997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.757075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.757236 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.757319 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.757470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.757555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.757708 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.757789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.757945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.758022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.758177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.758264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.758414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.758491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.758653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.758732 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.758889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.759057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.759132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.759202 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.759350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.759428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.759583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.759669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.759821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.759900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.760083 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.760162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.760341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.760418 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.760602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.760696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.760873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.760953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.761111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.external = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.761189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.761350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.761430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.761586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.761670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.761835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.761913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.762074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.762161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.password = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.762315 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] auth.token = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.762398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.762558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.762664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.762797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.762877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.763029 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.external = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.763107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.763262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.763340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.763507 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.763584 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.763760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.763839 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.763996 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.764074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.password = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.764226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] auth.token = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.764303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.764469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.764549 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.764733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.764826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.764984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.765062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.765217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.765295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.765462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.765539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.765702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.765780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.765935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.766020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.766172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.766256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.766419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.766496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.766668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.766747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.766915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.766992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.767155 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.767232 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.767405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.767491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.767642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.767728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.767894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.767972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.768142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.768220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.768375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.768452 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.768620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.768754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.768890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.768971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.769123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.769201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.769372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.769458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.769611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.769699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.769869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.769947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.770108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.770195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.770348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.770482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.770595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.770674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.770820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.770908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.771047 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.771124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.771293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.771370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.771527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.771613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.771766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.771843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.772003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.772081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.772253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.772340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.772486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.772564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.772766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.772848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.773005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.773162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.773239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.773314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.773478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.773556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.773711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.773796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.773942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.774019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.774187 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.774284 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.774402 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.774486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.774645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.774731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.774895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.774973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.775147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.775275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.775380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.775459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.775665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.775753 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.775902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.775990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.776136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.776214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.776378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.776456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.776613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.776721 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.776877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.776956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.777119 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.777197 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.777361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.777449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.777611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.777690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.777845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.777922 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.778072 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.778158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.778311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.778390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.778544 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.778622 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.778780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.778866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.779021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.779099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.779252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.779330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.779494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.779574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.779722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.779799 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.779959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.780036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.780200 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.780285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.780432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.780583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.780710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.780855 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.780946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.781102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.781180 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.781356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.781434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.781595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.781682 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.781831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.781898 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.782064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.782141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.782312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.782441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.782560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.782688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.782799 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.782930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.783038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.783181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.783260 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.783431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.783510 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.783665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.783743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.783899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.783979 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.784148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.784226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.784394 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.784472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.784634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.784733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.784893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.784972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.785132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.785210 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.785364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.785447 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.785602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.785680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.785840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.785918 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.786078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.786156 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.786310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.786386 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.786558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.786634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.786802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.786880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.787036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.787113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.787266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.787346 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.787501 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.787581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.787736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.787813 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.787973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.788052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.788204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.788286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.788444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.788521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.788707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.788788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.788947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.789027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.789189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.789268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.789433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.789511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.789665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.789745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.789912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.789989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.790143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.790219 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.790379 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.790458 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.790617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.790695 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.790852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.790928 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.791091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.791171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.791427 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.791510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.791687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.791765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.791923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.792004 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.792166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.792245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.792412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.792496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.792666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.792746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.792920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.792997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.793158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.793235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.793407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.793488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.793651 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.793729 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.793883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.793960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.794122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.794201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.794363 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.794441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.794601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.794686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.794841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] role.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.794919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.795073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.795150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] role.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.795306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.795392 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.795559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.795645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.795796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.795873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.796041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.796190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.796266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.796343 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.796507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.796586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.796768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] role.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.796861 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.797015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.797092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] role.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.797243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.797320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.797483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.797569 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.797729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.797806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.797973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.798050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.798216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.798301 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.798459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.798538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.798701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.798878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.798957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.799119 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.799198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.799372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.799450 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.799623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.799701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.799871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.799949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.800118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.800196 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.800354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.800432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.800614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.800786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.800895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.801024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.801141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.801278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.801378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.801527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.801605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.801776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.801854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.802027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.802107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.802262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.802341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.802522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.802600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.802776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.802855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.803013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.803090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.803243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.803320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.803471 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.803550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] token.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.803702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.803781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.803949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.804025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.804193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.804272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.804439 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.804516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.804699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.804781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.804940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.805025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.805169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.805247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.805406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.805484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.805646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.805726 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.805891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.806014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.806124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.806209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.806358 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.806446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.806604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.806691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.806879 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.806964 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.807135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.807226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.807383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.807495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.807628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.807769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.807854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.808040 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.connection = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.808135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.808293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.808379 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.808543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.808627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.808809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.808895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.809038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.809122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.809280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.809365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.809519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.809603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.809758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.809843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.connection = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.809999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.810094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.810247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.810332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.810487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.810574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.810734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.810819 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.810967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.811050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.811207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.811293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.811442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.811525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.811910 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.812000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.812000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.812127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.812198 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.812268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.812337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.812407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.812514 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 12:18:34.812613 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.812864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.813101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.813348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.813579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.813819 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.814052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:34.814223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 12:18:34.842675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.842944 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.843176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.843534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.843764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.843988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.851753 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.852508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.880951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.881153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.886619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.886862 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.934766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.934976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.984300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.984522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.984753 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:34.984997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.143361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.143677 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.143770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.143837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.143925 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.143989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.144056 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.144207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.144314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.144403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.144468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.144679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.167769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.167989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.168126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.168271 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.168395 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.168520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.168641 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.168793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.168928 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.169044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.169157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.169276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.169400 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.169535 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.180187 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.180484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.192973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.193115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.193333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.193474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.193544 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.193617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.193732 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.193821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.193984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.194077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.194140 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.194198 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.194395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.194617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:35.210838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.211088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 12:18:35.211210 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.211391 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.211517 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 12:18:35.211607 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 12:18:35.211777 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.211902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.211970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 12:18:35.212105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.212243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.212488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.212946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.213377 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.213500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.213853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.222738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.223027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.223294 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.223416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.223574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.223717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.223829 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.224140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.230655 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.231171 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.233566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.233856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.234077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.234177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.234389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.234512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.234671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.234796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.234942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.235064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.235198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.235296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.235474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.235561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.235733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.235821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.236021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.236109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.236375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.236460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.236678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.236978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.258957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.259629 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.260434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.260804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.261160 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.261447 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.265499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.265759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.266075 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.266198 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.266400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.266688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.272398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.272789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.273051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.273315 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.277117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.277412 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.277533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.277696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.277821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.277953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.278046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.278224 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.278347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.278485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.278592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.278789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.278914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.279007 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.279074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.279338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.296021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.296311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.296835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.297123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.300165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.301003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.303407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.303709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.304226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.304527 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.308128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.308822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.311261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.311963 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.314520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.314821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.315116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.315238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.315372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.315582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.315726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.315807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.315949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.316117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.316442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.316758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.332010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.332301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.332577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.332718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.332835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.332944 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.333166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.333434 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.340860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.341151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.342981 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.343260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.343486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.343966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.345922 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.346218 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.346512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.346812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.347094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.348498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.348899 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.349165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.349468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.349757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.358523 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.358812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.359128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.359410 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.359693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.360061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.360347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.361757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.362045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.362333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.362619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.362904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.363185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.363472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.375852 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.378905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.379140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.381816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.382200 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.383249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.384711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.385010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.385152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.385455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.386591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.388074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.388360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.388683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.391886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.393359 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.395139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.396348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.396676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.396819 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.396983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.397269 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.399578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.399882 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.400170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.400449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.405401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.405673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.405987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.408497 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.408819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.409103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.412854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.413147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.413430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.413702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.413945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.414245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.414514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.414802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.415086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.415351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.415626 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.415884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.415974 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:35.416142 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.416321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.416499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.416620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.416929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.417209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.417496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.417780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.418053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.418346 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.418628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.418905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.419171 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.419451 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.419723 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:35.453675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561cd7a1b9c0 pid: 81634 (default app) Aug 24 12:18:35.454433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561cd7a1b9c0 pid: 81635 (default app) Aug 24 12:18:35.457154 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688316d-c237-4dca-9a79-211689b4af12 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:35.457443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688316d-c237-4dca-9a79-211689b4af12 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:35.457712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688316d-c237-4dca-9a79-211689b4af12 None None] PATH_INFO: `/v3` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:35.461899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Wed Aug 24 12:18:35 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:42.075898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636ab47-56e7-4bf6-991e-497067f74f0a None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:42.076506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636ab47-56e7-4bf6-991e-497067f74f0a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:42.076506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636ab47-56e7-4bf6-991e-497067f74f0a None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:42.080818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:18:42.089275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb7ac5-5238-4532-98f4-1595d76c7642 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:42.089697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb7ac5-5238-4532-98f4-1595d76c7642 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:42.090132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb7ac5-5238-4532-98f4-1595d76c7642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:42.161914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: INFO dbcounter [None req-4cbb7ac5-5238-4532-98f4-1595d76c7642 None None] Registered counter for database keystone Aug 24 12:18:42.184571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG oslo_db.sqlalchemy.engines [None req-4cbb7ac5-5238-4532-98f4-1595d76c7642 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81634) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 12:18:42.196768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writer thread running {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 12:18:42.384889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG passlib.handlers.bcrypt [None req-4cbb7ac5-5238-4532-98f4-1595d76c7642 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81634) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 12:18:42.385298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG passlib.handlers.bcrypt [None req-4cbb7ac5-5238-4532-98f4-1595d76c7642 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81634) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 12:18:42.441281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4cbb7ac5-5238-4532-98f4-1595d76c7642 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:42.442887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4cbb7ac5-5238-4532-98f4-1595d76c7642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:42.581191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 12:18:43.118805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636ab47-56e7-4bf6-991e-497067f74f0a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:43.193878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: INFO dbcounter [None req-c636ab47-56e7-4bf6-991e-497067f74f0a None None] Registered counter for database keystone Aug 24 12:18:43.219038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG oslo_db.sqlalchemy.engines [None req-c636ab47-56e7-4bf6-991e-497067f74f0a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81635) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 12:18:43.227081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writer thread running {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 12:18:43.351885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636ab47-56e7-4bf6-991e-497067f74f0a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:43.357862 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba25879-2d70-4063-974e-5ebc6c73f022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:43.358488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba25879-2d70-4063-974e-5ebc6c73f022 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:43.358763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba25879-2d70-4063-974e-5ebc6c73f022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:43.359035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba25879-2d70-4063-974e-5ebc6c73f022 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:43.368389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba25879-2d70-4063-974e-5ebc6c73f022 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:43.419495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9ba25879-2d70-4063-974e-5ebc6c73f022 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 12:18:43.419835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba25879-2d70-4063-974e-5ebc6c73f022 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:43.430071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Wed Aug 24 12:18:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:18:43.437384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb7ac5-5238-4532-98f4-1595d76c7642 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:43.462019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb7ac5-5238-4532-98f4-1595d76c7642 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:43.468979 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29cce1-2cee-4bfe-8f33-4e65de785998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:43.470108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29cce1-2cee-4bfe-8f33-4e65de785998 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:43.470389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29cce1-2cee-4bfe-8f33-4e65de785998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:43.470669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29cce1-2cee-4bfe-8f33-4e65de785998 admin admin] PATH_INFO: `/v3/projects/66ac24841ce6483cbf8285ea30e957f8` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:43.472453 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29cce1-2cee-4bfe-8f33-4e65de785998 admin admin] RBAC: Authorizing `identity:get_project(project_id=66ac24841ce6483cbf8285ea30e957f8)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:43.525472 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7c29cce1-2cee-4bfe-8f33-4e65de785998 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 12:18:43.525787 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29cce1-2cee-4bfe-8f33-4e65de785998 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:43.530451 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Wed Aug 24 12:18:43 2022] GET /identity/v3/projects/66ac24841ce6483cbf8285ea30e957f8 => generated 344 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:45.416371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1888c31-9a66-4acf-b5a8-fc972eea071f None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.417383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1888c31-9a66-4acf-b5a8-fc972eea071f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.417950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1888c31-9a66-4acf-b5a8-fc972eea071f None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.422248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:45.428911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f977b39-f5ac-4964-b445-720bebeb60a6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.429360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f977b39-f5ac-4964-b445-720bebeb60a6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.429477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f977b39-f5ac-4964-b445-720bebeb60a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.457595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9f977b39-f5ac-4964-b445-720bebeb60a6 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:45.459148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9f977b39-f5ac-4964-b445-720bebeb60a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:45.468539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 12:18:45.746297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1888c31-9a66-4acf-b5a8-fc972eea071f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:45.760892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1888c31-9a66-4acf-b5a8-fc972eea071f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:45.765937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79aed1-f353-44a2-90f4-12104716fb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:45.766529 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79aed1-f353-44a2-90f4-12104716fb38 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.766815 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79aed1-f353-44a2-90f4-12104716fb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.767093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79aed1-f353-44a2-90f4-12104716fb38 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.775870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79aed1-f353-44a2-90f4-12104716fb38 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:45.777778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79aed1-f353-44a2-90f4-12104716fb38 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:45.786858 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Wed Aug 24 12:18:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:18:45.792416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f977b39-f5ac-4964-b445-720bebeb60a6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:45.802123 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f977b39-f5ac-4964-b445-720bebeb60a6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:45.806959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f9c92-b379-4096-a4b2-dc6c33a5d3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:45.807591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f9c92-b379-4096-a4b2-dc6c33a5d3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.807870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f9c92-b379-4096-a4b2-dc6c33a5d3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.808232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f9c92-b379-4096-a4b2-dc6c33a5d3a1 admin admin] PATH_INFO: `/v3/users/fe15d570e61b4d028bf4767532315b26` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.809859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f9c92-b379-4096-a4b2-dc6c33a5d3a1 admin admin] RBAC: Authorizing `identity:get_user(user_id=fe15d570e61b4d028bf4767532315b26)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:45.811732 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f9c92-b379-4096-a4b2-dc6c33a5d3a1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:45.813382 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Wed Aug 24 12:18:45 2022] GET /identity/v3/users/fe15d570e61b4d028bf4767532315b26 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:47.767606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec889fef-d38c-4c12-9bba-c1a409b566a0 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:47.768168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec889fef-d38c-4c12-9bba-c1a409b566a0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:47.768168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec889fef-d38c-4c12-9bba-c1a409b566a0 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:47.769504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:47.774822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323924d3-71fe-42f2-baad-15c2aa07b6aa None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:47.775256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323924d3-71fe-42f2-baad-15c2aa07b6aa None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:47.775432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323924d3-71fe-42f2-baad-15c2aa07b6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:47.795158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-323924d3-71fe-42f2-baad-15c2aa07b6aa None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:47.796348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-323924d3-71fe-42f2-baad-15c2aa07b6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:47.814392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de08703-4d09-4175-a903-1f084b86ea61 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:47.814748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de08703-4d09-4175-a903-1f084b86ea61 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:47.815087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de08703-4d09-4175-a903-1f084b86ea61 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:47.816442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:18:47.821513 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dc903-bcf1-47e7-9525-f88345d7cd0d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:47.821796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dc903-bcf1-47e7-9525-f88345d7cd0d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:47.822069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dc903-bcf1-47e7-9525-f88345d7cd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:47.866818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG passlib.handlers.bcrypt [None req-c45dc903-bcf1-47e7-9525-f88345d7cd0d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81635) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 12:18:47.867241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG passlib.handlers.bcrypt [None req-c45dc903-bcf1-47e7-9525-f88345d7cd0d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81635) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 12:18:47.886058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:47.912223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c45dc903-bcf1-47e7-9525-f88345d7cd0d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:47.913827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c45dc903-bcf1-47e7-9525-f88345d7cd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:48.015197 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:18:48.151950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323924d3-71fe-42f2-baad-15c2aa07b6aa None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.177036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:48.261967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323924d3-71fe-42f2-baad-15c2aa07b6aa None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.278097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dc903-bcf1-47e7-9525-f88345d7cd0d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.297513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae9476-d959-4026-befe-2471583f3492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.298151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9476-d959-4026-befe-2471583f3492 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.298431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9476-d959-4026-befe-2471583f3492 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.298740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9476-d959-4026-befe-2471583f3492 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.309858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae9476-d959-4026-befe-2471583f3492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-32ae9476-d959-4026-befe-2471583f3492 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:18:48.318617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:48.319541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:18:48.320370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:18:48.326013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae9476-d959-4026-befe-2471583f3492 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.389914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dc903-bcf1-47e7-9525-f88345d7cd0d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.392226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:48.410475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:48.427230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a803f2-b4c0-4387-b139-641860e12d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.428093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a803f2-b4c0-4387-b139-641860e12d9b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.428370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a803f2-b4c0-4387-b139-641860e12d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.428520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae9476-d959-4026-befe-2471583f3492 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.428671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a803f2-b4c0-4387-b139-641860e12d9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.464542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7094d9-473b-47c5-9f6f-d18db44d46de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.465944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7094d9-473b-47c5-9f6f-d18db44d46de None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.466281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7094d9-473b-47c5-9f6f-d18db44d46de None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.466555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7094d9-473b-47c5-9f6f-d18db44d46de None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.467235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7094d9-473b-47c5-9f6f-d18db44d46de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:48.467534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7094d9-473b-47c5-9f6f-d18db44d46de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:48.468979 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7094d9-473b-47c5-9f6f-d18db44d46de None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:48.482874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:48.489222 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7094d9-473b-47c5-9f6f-d18db44d46de None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.500488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a803f2-b4c0-4387-b139-641860e12d9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:48.502104 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a803f2-b4c0-4387-b139-641860e12d9b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:48.590432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7094d9-473b-47c5-9f6f-d18db44d46de None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.602389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 8/16] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:18:48.609009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a803f2-b4c0-4387-b139-641860e12d9b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.611852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:48.628536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e466e52-771a-4a61-aae7-5dcce4eac277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.629199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e466e52-771a-4a61-aae7-5dcce4eac277 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.629481 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e466e52-771a-4a61-aae7-5dcce4eac277 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.629753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e466e52-771a-4a61-aae7-5dcce4eac277 None admin] PATH_INFO: `/v3/users/fe15d570e61b4d028bf4767532315b26` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.630904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e466e52-771a-4a61-aae7-5dcce4eac277 None admin] RBAC: Authorizing `identity:get_user(user_id=fe15d570e61b4d028bf4767532315b26)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:48.632396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e466e52-771a-4a61-aae7-5dcce4eac277 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:48.634043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/users/fe15d570e61b4d028bf4767532315b26 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:48.640566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e466e52-771a-4a61-aae7-5dcce4eac277 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.657212 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:48.711554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a803f2-b4c0-4387-b139-641860e12d9b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.746455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e466e52-771a-4a61-aae7-5dcce4eac277 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.747445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dbcca-424e-49e5-b64e-a55a868179a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.748305 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dbcca-424e-49e5-b64e-a55a868179a2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.748563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dbcca-424e-49e5-b64e-a55a868179a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.748880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dbcca-424e-49e5-b64e-a55a868179a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.768249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dbcca-424e-49e5-b64e-a55a868179a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:48.769785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dbcca-424e-49e5-b64e-a55a868179a2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:48.780803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea80cad-a778-45c8-9e47-e0eee1df7340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.781445 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea80cad-a778-45c8-9e47-e0eee1df7340 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.781721 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea80cad-a778-45c8-9e47-e0eee1df7340 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.781985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea80cad-a778-45c8-9e47-e0eee1df7340 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.783263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea80cad-a778-45c8-9e47-e0eee1df7340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:48.784823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea80cad-a778-45c8-9e47-e0eee1df7340 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:48.786383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 10/18] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-323dbcca-424e-49e5-b64e-a55a868179a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:18:48.788370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:48.789287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:18:48.790096 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:18:48.790096 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:18:48.790096 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:18:48.790096 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 9/19] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:18:48.792255 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea80cad-a778-45c8-9e47-e0eee1df7340 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.795581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dbcca-424e-49e5-b64e-a55a868179a2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.851889 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:48.877044 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:48.896126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea80cad-a778-45c8-9e47-e0eee1df7340 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.900133 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dbcca-424e-49e5-b64e-a55a868179a2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.930531 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d180818-1af0-401e-953f-96a0f8e02738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.931141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d180818-1af0-401e-953f-96a0f8e02738 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.931420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d180818-1af0-401e-953f-96a0f8e02738 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.931667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d180818-1af0-401e-953f-96a0f8e02738 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.932397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d180818-1af0-401e-953f-96a0f8e02738 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.domain.id=default, user.id=fe15d570e61b4d028bf4767532315b26) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:48.932730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d180818-1af0-401e-953f-96a0f8e02738 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:48.934387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d180818-1af0-401e-953f-96a0f8e02738 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:48.941194 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 11/20] 10.222.0.27 () {56 vars in 1322 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/role_assignments?user.id=fe15d570e61b4d028bf4767532315b26&scope.domain.id=default&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:48.945607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f2867-9bc1-41e5-af82-543d5c4a7737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.946240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f2867-9bc1-41e5-af82-543d5c4a7737 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.946520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f2867-9bc1-41e5-af82-543d5c4a7737 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.946793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f2867-9bc1-41e5-af82-543d5c4a7737 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.947467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f2867-9bc1-41e5-af82-543d5c4a7737 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:48.947774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f2867-9bc1-41e5-af82-543d5c4a7737 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:48.949302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f2867-9bc1-41e5-af82-543d5c4a7737 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:48.972080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 10/21] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:51.080189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b01d1-e905-4b1d-b7d3-f723ced1b652 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:51.080743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b01d1-e905-4b1d-b7d3-f723ced1b652 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:51.080743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b01d1-e905-4b1d-b7d3-f723ced1b652 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:51.082399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 12/22] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:51.087308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a14991-137f-49f9-b514-e5c01d2edb45 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:51.087702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a14991-137f-49f9-b514-e5c01d2edb45 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:51.087868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a14991-137f-49f9-b514-e5c01d2edb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:51.122072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-08a14991-137f-49f9-b514-e5c01d2edb45 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:51.123300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-08a14991-137f-49f9-b514-e5c01d2edb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:51.225225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 11/23] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:51.243560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10150e8a-78f0-4a80-9a93-cce15d93f3e8 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:51.243972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10150e8a-78f0-4a80-9a93-cce15d93f3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:51.244458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10150e8a-78f0-4a80-9a93-cce15d93f3e8 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:51.247323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 13/24] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:51.253360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b41ea-a8b4-4061-9004-0acc2bf0b51d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:51.254079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b41ea-a8b4-4061-9004-0acc2bf0b51d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:51.254791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b41ea-a8b4-4061-9004-0acc2bf0b51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:51.291876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c93b41ea-a8b4-4061-9004-0acc2bf0b51d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:51.294432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c93b41ea-a8b4-4061-9004-0acc2bf0b51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:51.384385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:51.443252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 12/25] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:51.801489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10150e8a-78f0-4a80-9a93-cce15d93f3e8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:51.903357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b41ea-a8b4-4061-9004-0acc2bf0b51d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:51.946448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10150e8a-78f0-4a80-9a93-cce15d93f3e8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:51.992985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fbe61-d957-47f8-b495-3f6e26af44dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:51.994041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fbe61-d957-47f8-b495-3f6e26af44dc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:51.994513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fbe61-d957-47f8-b495-3f6e26af44dc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:51.995016 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fbe61-d957-47f8-b495-3f6e26af44dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:52.005882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fbe61-d957-47f8-b495-3f6e26af44dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e62fbe61-d957-47f8-b495-3f6e26af44dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:18:52.011634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:52.012447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:52.013220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:52.013220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:52.013220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:52.013220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:52.013220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:18:52.013220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:18:52.013220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:18:52.013220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:18:52.013220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:18:52.013220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:18:52.013220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:52.013220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:18:52.015129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 14/26] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:18:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:18:52.025656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fbe61-d957-47f8-b495-3f6e26af44dc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:52.056042 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:52.190367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b41ea-a8b4-4061-9004-0acc2bf0b51d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:52.213980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fbe61-d957-47f8-b495-3f6e26af44dc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:52.228462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199a213-cfb3-418d-b0c8-2a336be46422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:52.229452 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199a213-cfb3-418d-b0c8-2a336be46422 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:52.229945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199a213-cfb3-418d-b0c8-2a336be46422 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:52.231044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199a213-cfb3-418d-b0c8-2a336be46422 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:52.241698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199a213-cfb3-418d-b0c8-2a336be46422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:52.243413 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199a213-cfb3-418d-b0c8-2a336be46422 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-c199a213-cfb3-418d-b0c8-2a336be46422 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:18:52.258731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:52.259593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:18:52.260369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:18:52.260369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:18:52.260369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:18:52.262946 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 13/27] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:18:52.264397 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259ff34-7952-4a7d-bc20-c4471000de91 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:52.265065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259ff34-7952-4a7d-bc20-c4471000de91 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:52.265542 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259ff34-7952-4a7d-bc20-c4471000de91 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:52.267376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 14/28] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:52.271545 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a08caa-6087-4eea-8132-a5e773a7db36 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:52.272392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a08caa-6087-4eea-8132-a5e773a7db36 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:52.273224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a08caa-6087-4eea-8132-a5e773a7db36 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:52.275773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 15/29] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:52.277170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a08caa-6087-4eea-8132-a5e773a7db36 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:52.289781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cc6c8-7101-44ca-8d38-6bf0dc148e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:52.290817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cc6c8-7101-44ca-8d38-6bf0dc148e00 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:52.291313 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cc6c8-7101-44ca-8d38-6bf0dc148e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:52.291798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cc6c8-7101-44ca-8d38-6bf0dc148e00 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:52.293525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cc6c8-7101-44ca-8d38-6bf0dc148e00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:52.293767 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cc6c8-7101-44ca-8d38-6bf0dc148e00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:52.296724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cc6c8-7101-44ca-8d38-6bf0dc148e00 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:52.312279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:52.316486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 15/30] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:52.319035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c97091-1302-4705-8e1c-2aa00ff1057b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:52.319982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c97091-1302-4705-8e1c-2aa00ff1057b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:52.321217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c97091-1302-4705-8e1c-2aa00ff1057b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:52.355348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-15c97091-1302-4705-8e1c-2aa00ff1057b None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:52.357853 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-15c97091-1302-4705-8e1c-2aa00ff1057b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:52.393156 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a08caa-6087-4eea-8132-a5e773a7db36 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:52.435170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ea8d5-c5bd-40fc-b4e3-2837fdd7ca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:52.435953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ea8d5-c5bd-40fc-b4e3-2837fdd7ca7c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:52.436355 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ea8d5-c5bd-40fc-b4e3-2837fdd7ca7c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:52.436760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ea8d5-c5bd-40fc-b4e3-2837fdd7ca7c None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:52.437516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ea8d5-c5bd-40fc-b4e3-2837fdd7ca7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:52.437919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ea8d5-c5bd-40fc-b4e3-2837fdd7ca7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:52.439444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ea8d5-c5bd-40fc-b4e3-2837fdd7ca7c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:52.456609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 16/31] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:52.458134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e6b73-4007-43fe-a761-bf244bb7f5b1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:52.458522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e6b73-4007-43fe-a761-bf244bb7f5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:52.458900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e6b73-4007-43fe-a761-bf244bb7f5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:52.478089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b91e6b73-4007-43fe-a761-bf244bb7f5b1 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:52.478553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:52.479635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b91e6b73-4007-43fe-a761-bf244bb7f5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:52.495907 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 16/32] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:52.497864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c97091-1302-4705-8e1c-2aa00ff1057b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:52.553934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 17/33] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:52.554855 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e6b73-4007-43fe-a761-bf244bb7f5b1 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:52.573783 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:52.667544 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e6b73-4007-43fe-a761-bf244bb7f5b1 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:52.699441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdb4ef-6d7c-4cd6-84df-7fc047e350b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:52.700389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdb4ef-6d7c-4cd6-84df-7fc047e350b0 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:52.700687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdb4ef-6d7c-4cd6-84df-7fc047e350b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:52.700974 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdb4ef-6d7c-4cd6-84df-7fc047e350b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:52.702314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdb4ef-6d7c-4cd6-84df-7fc047e350b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:52.703331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdb4ef-6d7c-4cd6-84df-7fc047e350b0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:52.768544 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c97091-1302-4705-8e1c-2aa00ff1057b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:52.843311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323030ca-a338-44ef-a07d-5c7ad739fd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:52.843923 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323030ca-a338-44ef-a07d-5c7ad739fd82 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:52.844198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323030ca-a338-44ef-a07d-5c7ad739fd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:52.844482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323030ca-a338-44ef-a07d-5c7ad739fd82 None admin] PATH_INFO: `/v3/users/fe15d570e61b4d028bf4767532315b26` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:52.849061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 18/34] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:18:52 2022] POST /identity/v3/projects => generated 301 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:18:52.849391 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323030ca-a338-44ef-a07d-5c7ad739fd82 None admin] RBAC: Authorizing `identity:get_user(user_id=fe15d570e61b4d028bf4767532315b26)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:52.850862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323030ca-a338-44ef-a07d-5c7ad739fd82 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:52.852750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 17/35] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:18:52 2022] GET /identity/v3/users/fe15d570e61b4d028bf4767532315b26 => generated 247 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:52.858532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdb4ef-6d7c-4cd6-84df-7fc047e350b0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:52.962465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:53.023188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdb4ef-6d7c-4cd6-84df-7fc047e350b0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:53.081556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e82b9-5b5c-4019-8c5c-a17b53fec9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:53.082625 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e82b9-5b5c-4019-8c5c-a17b53fec9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:53.083164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e82b9-5b5c-4019-8c5c-a17b53fec9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:53.083624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e82b9-5b5c-4019-8c5c-a17b53fec9a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:53.085327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e82b9-5b5c-4019-8c5c-a17b53fec9a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:53.087154 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e82b9-5b5c-4019-8c5c-a17b53fec9a7 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:53.089355 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 19/36] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:53.095420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323030ca-a338-44ef-a07d-5c7ad739fd82 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:53.104469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f9999-e588-4ff8-93b0-d620306459a2 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:53.105155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f9999-e588-4ff8-93b0-d620306459a2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:53.105640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f9999-e588-4ff8-93b0-d620306459a2 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:53.108199 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 20/37] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:53.114484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4543a7-e631-4f4b-9643-eef45840c68d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:53.115474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4543a7-e631-4f4b-9643-eef45840c68d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:53.115941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4543a7-e631-4f4b-9643-eef45840c68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:53.138460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5a4543a7-e631-4f4b-9643-eef45840c68d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:53.140394 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5a4543a7-e631-4f4b-9643-eef45840c68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:53.213124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:53.238725 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 21/38] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:18:53.239580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4543a7-e631-4f4b-9643-eef45840c68d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:53.264213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323030ca-a338-44ef-a07d-5c7ad739fd82 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:53.309738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2fe58-0e2f-47c2-b52b-c9a3f2944640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:53.310404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2fe58-0e2f-47c2-b52b-c9a3f2944640 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:53.310681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2fe58-0e2f-47c2-b52b-c9a3f2944640 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:53.310953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2fe58-0e2f-47c2-b52b-c9a3f2944640 None admin] PATH_INFO: `/v3/domains/default/users/fe15d570e61b4d028bf4767532315b26/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:53.312937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2fe58-0e2f-47c2-b52b-c9a3f2944640 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fe15d570e61b4d028bf4767532315b26, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:53.314996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2fe58-0e2f-47c2-b52b-c9a3f2944640 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:53.322763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:53.356694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4543a7-e631-4f4b-9643-eef45840c68d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:53.391841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747043a-a932-4a69-894a-989a2065b920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:53.392451 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747043a-a932-4a69-894a-989a2065b920 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:53.392757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747043a-a932-4a69-894a-989a2065b920 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:53.393038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747043a-a932-4a69-894a-989a2065b920 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:53.393702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747043a-a932-4a69-894a-989a2065b920 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:53.395102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747043a-a932-4a69-894a-989a2065b920 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:53.413677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 22/39] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 24 12:18:53 2022] POST /identity/v3/roles => generated 219 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:18:53.414560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747043a-a932-4a69-894a-989a2065b920 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:53.432875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 18/40] 10.222.0.27 () {58 vars in 1361 bytes} [Wed Aug 24 12:18:53 2022] PUT /identity/v3/domains/default/users/fe15d570e61b4d028bf4767532315b26/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:18:53.435427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a39d27-bf2b-46be-baba-bcb4336b669b None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:53.436022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a39d27-bf2b-46be-baba-bcb4336b669b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:53.436519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a39d27-bf2b-46be-baba-bcb4336b669b None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:53.446346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 19/41] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:53 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:53.448149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13c49d-cfdc-48de-961c-f17d97e302db None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:53.448717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13c49d-cfdc-48de-961c-f17d97e302db None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:53.449247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13c49d-cfdc-48de-961c-f17d97e302db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:53.472173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9e13c49d-cfdc-48de-961c-f17d97e302db None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:53.474278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9e13c49d-cfdc-48de-961c-f17d97e302db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:53.573974 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 20/42] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:53.575315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13c49d-cfdc-48de-961c-f17d97e302db None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:53.619679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747043a-a932-4a69-894a-989a2065b920 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:53.639374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:53.651292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:53.662240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708c260-4e30-49a9-a30f-6effde65e3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:53.662865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708c260-4e30-49a9-a30f-6effde65e3d0 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:53.663151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708c260-4e30-49a9-a30f-6effde65e3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:53.663412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708c260-4e30-49a9-a30f-6effde65e3d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:53.664073 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708c260-4e30-49a9-a30f-6effde65e3d0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:53.666576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708c260-4e30-49a9-a30f-6effde65e3d0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:53.689230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 23/43] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 24 12:18:53 2022] POST /identity/v3/roles => generated 225 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:18:53.692225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13c49d-cfdc-48de-961c-f17d97e302db None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:53.733490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4fe4c-d897-4344-ad3c-15a41778c132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:53.734117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4fe4c-d897-4344-ad3c-15a41778c132 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:53.734397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4fe4c-d897-4344-ad3c-15a41778c132 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:53.734671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4fe4c-d897-4344-ad3c-15a41778c132 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:53.735407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4fe4c-d897-4344-ad3c-15a41778c132 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:53.737011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4fe4c-d897-4344-ad3c-15a41778c132 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:53.758030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 21/44] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 24 12:18:53 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:18:53.888425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708c260-4e30-49a9-a30f-6effde65e3d0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:54.009127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708c260-4e30-49a9-a30f-6effde65e3d0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:54.151801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8f341-b890-43af-a55a-4e31ad51f30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:54.152389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8f341-b890-43af-a55a-4e31ad51f30b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:54.152680 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8f341-b890-43af-a55a-4e31ad51f30b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:54.152975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8f341-b890-43af-a55a-4e31ad51f30b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:54.154191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8f341-b890-43af-a55a-4e31ad51f30b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:54.156236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8f341-b890-43af-a55a-4e31ad51f30b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:54.158065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 24/45] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:54.163605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4fe4c-d897-4344-ad3c-15a41778c132 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:54.272876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4fe4c-d897-4344-ad3c-15a41778c132 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:54.287035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:54.310458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dddc2c2-99bf-46ac-afb9-bd888dea439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:54.311082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dddc2c2-99bf-46ac-afb9-bd888dea439f None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:54.311365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dddc2c2-99bf-46ac-afb9-bd888dea439f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:54.311636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dddc2c2-99bf-46ac-afb9-bd888dea439f None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:54.312287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dddc2c2-99bf-46ac-afb9-bd888dea439f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:54.313794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dddc2c2-99bf-46ac-afb9-bd888dea439f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:54.454134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 22/46] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:18:54 2022] POST /identity/v3/projects => generated 312 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:18:55.438251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae43508-a90c-4b39-8aa0-c8c729a55a9a None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.438785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae43508-a90c-4b39-8aa0-c8c729a55a9a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.438785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae43508-a90c-4b39-8aa0-c8c729a55a9a None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.440320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 25/47] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:55.445554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b4ea-eeac-424e-96af-cdcccdf81534 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.445989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b4ea-eeac-424e-96af-cdcccdf81534 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.446110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b4ea-eeac-424e-96af-cdcccdf81534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.466673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ce55b4ea-eeac-424e-96af-cdcccdf81534 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:55.467890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ce55b4ea-eeac-424e-96af-cdcccdf81534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:55.545637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 23/48] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:55.811851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae43508-a90c-4b39-8aa0-c8c729a55a9a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:55.874029 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:55.923425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae43508-a90c-4b39-8aa0-c8c729a55a9a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:55.960202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee345a-1ea2-430d-9939-f1ed31f387b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:55.960869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee345a-1ea2-430d-9939-f1ed31f387b8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.961144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee345a-1ea2-430d-9939-f1ed31f387b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.961415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee345a-1ea2-430d-9939-f1ed31f387b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.973437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee345a-1ea2-430d-9939-f1ed31f387b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-02ee345a-1ea2-430d-9939-f1ed31f387b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:18:55.982834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:55.983779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:18:55.984602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 26/49] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:18:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:18:55.991036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b4ea-eeac-424e-96af-cdcccdf81534 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.091658 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:56.097989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b4ea-eeac-424e-96af-cdcccdf81534 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.134755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad0294-2b43-4981-b55c-cdf27e25b313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.135360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad0294-2b43-4981-b55c-cdf27e25b313 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.135635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad0294-2b43-4981-b55c-cdf27e25b313 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.135903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad0294-2b43-4981-b55c-cdf27e25b313 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.136596 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad0294-2b43-4981-b55c-cdf27e25b313 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:56.137294 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad0294-2b43-4981-b55c-cdf27e25b313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.138314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad0294-2b43-4981-b55c-cdf27e25b313 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.146632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 24/50] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:18:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.152550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee345a-1ea2-430d-9939-f1ed31f387b8 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.257125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee345a-1ea2-430d-9939-f1ed31f387b8 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.274081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:56.293141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130d9da-41ce-45f1-a202-82c219d0016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.293864 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130d9da-41ce-45f1-a202-82c219d0016d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.294148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130d9da-41ce-45f1-a202-82c219d0016d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.294424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130d9da-41ce-45f1-a202-82c219d0016d None admin] PATH_INFO: `/v3/users/fe15d570e61b4d028bf4767532315b26` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.295813 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130d9da-41ce-45f1-a202-82c219d0016d None admin] RBAC: Authorizing `identity:get_user(user_id=fe15d570e61b4d028bf4767532315b26)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.297332 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130d9da-41ce-45f1-a202-82c219d0016d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.298969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 27/51] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/users/fe15d570e61b4d028bf4767532315b26 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.304612 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad0294-2b43-4981-b55c-cdf27e25b313 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.347164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153db9b2-134f-4c62-bec0-169d689250c9 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.347459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153db9b2-134f-4c62-bec0-169d689250c9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.347733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153db9b2-134f-4c62-bec0-169d689250c9 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.349132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 28/52] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:18:56.354224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569ae89-c219-4400-9e69-79c6753612e7 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.354510 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569ae89-c219-4400-9e69-79c6753612e7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.354788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569ae89-c219-4400-9e69-79c6753612e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.373835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8569ae89-c219-4400-9e69-79c6753612e7 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:56.375040 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8569ae89-c219-4400-9e69-79c6753612e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:56.408985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad0294-2b43-4981-b55c-cdf27e25b313 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.447143 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a31db-0ab8-4979-89e7-d984213c96a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.447793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a31db-0ab8-4979-89e7-d984213c96a4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.448042 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a31db-0ab8-4979-89e7-d984213c96a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.448308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a31db-0ab8-4979-89e7-d984213c96a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.449606 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a31db-0ab8-4979-89e7-d984213c96a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.451069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a31db-0ab8-4979-89e7-d984213c96a4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.451784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 29/53] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:18:56.452658 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 25/54] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.458281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569ae89-c219-4400-9e69-79c6753612e7 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.561793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569ae89-c219-4400-9e69-79c6753612e7 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.576018 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:56.598691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b3946-44fe-4f39-9dd7-bd0e14a8a279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.599234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b3946-44fe-4f39-9dd7-bd0e14a8a279 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.599521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b3946-44fe-4f39-9dd7-bd0e14a8a279 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.599802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b3946-44fe-4f39-9dd7-bd0e14a8a279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.600527 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b3946-44fe-4f39-9dd7-bd0e14a8a279 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.domain.id=default, user.id=fe15d570e61b4d028bf4767532315b26) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:56.600878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b3946-44fe-4f39-9dd7-bd0e14a8a279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.602384 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b3946-44fe-4f39-9dd7-bd0e14a8a279 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.611635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 30/55] 10.222.0.27 () {56 vars in 1322 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/role_assignments?user.id=fe15d570e61b4d028bf4767532315b26&scope.domain.id=default&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.719684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a31db-0ab8-4979-89e7-d984213c96a4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.752540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:56.830452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a31db-0ab8-4979-89e7-d984213c96a4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.867099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb6b5c-fc13-433f-aa06-a8a3f1277c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.867669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb6b5c-fc13-433f-aa06-a8a3f1277c47 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.867951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb6b5c-fc13-433f-aa06-a8a3f1277c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.868225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb6b5c-fc13-433f-aa06-a8a3f1277c47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.869363 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb6b5c-fc13-433f-aa06-a8a3f1277c47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.870852 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb6b5c-fc13-433f-aa06-a8a3f1277c47 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.872707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 26/56] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.879133 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b3946-44fe-4f39-9dd7-bd0e14a8a279 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:57.007406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b3946-44fe-4f39-9dd7-bd0e14a8a279 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:57.057479 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27415f65-4f67-408e-bd22-95c50bb9daa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:57.058451 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27415f65-4f67-408e-bd22-95c50bb9daa8 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:57.058897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27415f65-4f67-408e-bd22-95c50bb9daa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:57.059160 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27415f65-4f67-408e-bd22-95c50bb9daa8 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:57.059806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27415f65-4f67-408e-bd22-95c50bb9daa8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:57.061329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27415f65-4f67-408e-bd22-95c50bb9daa8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:57.095951 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 31/57] 10.222.0.27 () {60 vars in 1140 bytes} [Wed Aug 24 12:18:56 2022] POST /identity/v3/projects => generated 298 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:18:59.033569 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d36a3d-9417-4007-ba94-6c3a50817025 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:59.034361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d36a3d-9417-4007-ba94-6c3a50817025 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:59.034532 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d36a3d-9417-4007-ba94-6c3a50817025 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:59.036626 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 27/58] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:18:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:59.042189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12404b7-4645-4a0a-84c0-a2115854d336 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:59.042602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12404b7-4645-4a0a-84c0-a2115854d336 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:59.042727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12404b7-4645-4a0a-84c0-a2115854d336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:59.066962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a12404b7-4645-4a0a-84c0-a2115854d336 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:59.068185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a12404b7-4645-4a0a-84c0-a2115854d336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:59.096863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:59.143911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 32/59] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:59.423183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d36a3d-9417-4007-ba94-6c3a50817025 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:59.536696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:59.545269 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d36a3d-9417-4007-ba94-6c3a50817025 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:59.582036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca4e62-f912-4b40-9579-b854d1490443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:59.582662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca4e62-f912-4b40-9579-b854d1490443 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:59.582942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca4e62-f912-4b40-9579-b854d1490443 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:59.583219 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca4e62-f912-4b40-9579-b854d1490443 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:59.584466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca4e62-f912-4b40-9579-b854d1490443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:59.585987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca4e62-f912-4b40-9579-b854d1490443 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:59.587574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 28/60] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:59.595712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12404b7-4645-4a0a-84c0-a2115854d336 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:59.700453 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12404b7-4645-4a0a-84c0-a2115854d336 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:59.736089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f934572-501a-43ac-8d84-57d3df6eaa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:59.736688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f934572-501a-43ac-8d84-57d3df6eaa9f None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:59.736977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f934572-501a-43ac-8d84-57d3df6eaa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:59.737244 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f934572-501a-43ac-8d84-57d3df6eaa9f None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:59.737872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f934572-501a-43ac-8d84-57d3df6eaa9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:59.739328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f934572-501a-43ac-8d84-57d3df6eaa9f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:59.773784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 33/61] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:18:59 2022] POST /identity/v3/users => generated 275 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:02.794394 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4cbfb-b227-4812-9860-100316d6f049 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:02.795402 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4cbfb-b227-4812-9860-100316d6f049 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:02.796073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4cbfb-b227-4812-9860-100316d6f049 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:02.797684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 29/62] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:02.807820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f5d48-61d8-4963-93e6-fd036c88c5bf None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:02.808758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f5d48-61d8-4963-93e6-fd036c88c5bf None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:02.809246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f5d48-61d8-4963-93e6-fd036c88c5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:02.833825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ad8f5d48-61d8-4963-93e6-fd036c88c5bf None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:02.835036 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ad8f5d48-61d8-4963-93e6-fd036c88c5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:02.850568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:02.931452 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 34/63] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 12:19:03.102060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989eabd-85fa-446e-91dd-caaa42642092 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.102712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989eabd-85fa-446e-91dd-caaa42642092 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.103204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989eabd-85fa-446e-91dd-caaa42642092 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.104764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 30/64] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:03.110485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2ceb0-b685-40c1-9320-a4f9c7f0de20 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.110779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2ceb0-b685-40c1-9320-a4f9c7f0de20 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.111059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2ceb0-b685-40c1-9320-a4f9c7f0de20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.134426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f3a2ceb0-b685-40c1-9320-a4f9c7f0de20 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:03.135614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f3a2ceb0-b685-40c1-9320-a4f9c7f0de20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:03.234988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 35/65] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:03.237428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d6763-d9ac-4bb3-b37c-5c086f2c72a7 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.237916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d6763-d9ac-4bb3-b37c-5c086f2c72a7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.238392 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d6763-d9ac-4bb3-b37c-5c086f2c72a7 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.239913 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 31/66] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:03.245461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8589f0-3c47-4de9-bfba-78954bdf5bf4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.245752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8589f0-3c47-4de9-bfba-78954bdf5bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.246057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8589f0-3c47-4de9-bfba-78954bdf5bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.275419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2b8589f0-3c47-4de9-bfba-78954bdf5bf4 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:03.276727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2b8589f0-3c47-4de9-bfba-78954bdf5bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:03.289708 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d6763-d9ac-4bb3-b37c-5c086f2c72a7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:03.366555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 36/67] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:03.368242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bf65a-383b-40d0-9a91-52b99c1f2c13 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.368751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bf65a-383b-40d0-9a91-52b99c1f2c13 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.369232 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bf65a-383b-40d0-9a91-52b99c1f2c13 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.370774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 37/68] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:03.377937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfb23f-a12e-43e3-bf88-e5681b1c603f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.378467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfb23f-a12e-43e3-bf88-e5681b1c603f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.378977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfb23f-a12e-43e3-bf88-e5681b1c603f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.400329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:03.412731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a2dfb23f-a12e-43e3-bf88-e5681b1c603f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:03.413906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a2dfb23f-a12e-43e3-bf88-e5681b1c603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:03.441254 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d6763-d9ac-4bb3-b37c-5c086f2c72a7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:03.478449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b49d5f-1978-4a40-943a-36fe00d6732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:03.479368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b49d5f-1978-4a40-943a-36fe00d6732f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.479843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b49d5f-1978-4a40-943a-36fe00d6732f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.480379 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b49d5f-1978-4a40-943a-36fe00d6732f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.491621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b49d5f-1978-4a40-943a-36fe00d6732f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:03.494175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 38/69] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:03.495564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732469ef-c840-4053-8e23-50d88b01e994 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.495915 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732469ef-c840-4053-8e23-50d88b01e994 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.496293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732469ef-c840-4053-8e23-50d88b01e994 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.497879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 39/70] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c8b49d5f-1978-4a40-943a-36fe00d6732f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:03.502037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:03.503182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:03.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 32/71] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:03.506140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969c304-43ea-44bd-9f0e-f37f9bf11840 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.506420 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969c304-43ea-44bd-9f0e-f37f9bf11840 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.506760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969c304-43ea-44bd-9f0e-f37f9bf11840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.507758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b49d5f-1978-4a40-943a-36fe00d6732f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:03.529237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5969c304-43ea-44bd-9f0e-f37f9bf11840 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:03.530618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5969c304-43ea-44bd-9f0e-f37f9bf11840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:03.535190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:03.633187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 40/72] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:03.633994 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969c304-43ea-44bd-9f0e-f37f9bf11840 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:03.658359 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b49d5f-1978-4a40-943a-36fe00d6732f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:03.694955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eade1d4-cdc7-4551-9429-c38ade996010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:03.695561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eade1d4-cdc7-4551-9429-c38ade996010 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.695838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eade1d4-cdc7-4551-9429-c38ade996010 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.696108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eade1d4-cdc7-4551-9429-c38ade996010 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.696814 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eade1d4-cdc7-4551-9429-c38ade996010 None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:03.697117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eade1d4-cdc7-4551-9429-c38ade996010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:03.698511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eade1d4-cdc7-4551-9429-c38ade996010 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:03.707429 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 33/73] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:03.708232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eade1d4-cdc7-4551-9429-c38ade996010 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:03.709048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:03.750526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969c304-43ea-44bd-9f0e-f37f9bf11840 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:03.788455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020cbebe-2068-49d1-b0ff-cd7b9d39f226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:03.789215 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cbebe-2068-49d1-b0ff-cd7b9d39f226 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.789498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cbebe-2068-49d1-b0ff-cd7b9d39f226 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.789771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cbebe-2068-49d1-b0ff-cd7b9d39f226 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.795201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:03.802360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020cbebe-2068-49d1-b0ff-cd7b9d39f226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-020cbebe-2068-49d1-b0ff-cd7b9d39f226 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:03.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:03.811400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 41/74] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:03.813574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020cbebe-2068-49d1-b0ff-cd7b9d39f226 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:03.815354 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eade1d4-cdc7-4551-9429-c38ade996010 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:03.851483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffcf59-a422-42c6-855c-882caf40b403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:03.852111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffcf59-a422-42c6-855c-882caf40b403 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.852390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffcf59-a422-42c6-855c-882caf40b403 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.852688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffcf59-a422-42c6-855c-882caf40b403 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.863327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bffcf59-a422-42c6-855c-882caf40b403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-8bffcf59-a422-42c6-855c-882caf40b403 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:03.869466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:03.870299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:03.871073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:03.871073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:03.871073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:03.871073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:03.871073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:03.871073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:03.871073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:03.871073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:03.871073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:03.871073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:03.871073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:03.871073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:03.872221 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 34/75] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:03.872626 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffcf59-a422-42c6-855c-882caf40b403 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:03.923860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020cbebe-2068-49d1-b0ff-cd7b9d39f226 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:03.951984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:03.962310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7360a-3ebc-4b3f-b5a8-82626f474aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:03.962944 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7360a-3ebc-4b3f-b5a8-82626f474aee None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.963226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7360a-3ebc-4b3f-b5a8-82626f474aee None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.963498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7360a-3ebc-4b3f-b5a8-82626f474aee None admin] PATH_INFO: `/v3/users/8563b95970a84781849db7018a225599` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.982416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffcf59-a422-42c6-855c-882caf40b403 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:03.982632 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7360a-3ebc-4b3f-b5a8-82626f474aee None admin] RBAC: Authorizing `identity:get_user(user_id=8563b95970a84781849db7018a225599)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:03.984131 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7360a-3ebc-4b3f-b5a8-82626f474aee None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:03.985871 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 42/76] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:03 2022] GET /identity/v3/users/8563b95970a84781849db7018a225599 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:03.986644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7360a-3ebc-4b3f-b5a8-82626f474aee None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.018963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c282358-01b2-4d71-8ab1-88af61f6235b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.019602 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c282358-01b2-4d71-8ab1-88af61f6235b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.019877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c282358-01b2-4d71-8ab1-88af61f6235b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.020149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c282358-01b2-4d71-8ab1-88af61f6235b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.031906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c282358-01b2-4d71-8ab1-88af61f6235b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-5c282358-01b2-4d71-8ab1-88af61f6235b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:04.036683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:04.037534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:04.038594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 35/77] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:04.039418 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c282358-01b2-4d71-8ab1-88af61f6235b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.043654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:04.091429 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7360a-3ebc-4b3f-b5a8-82626f474aee None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.128698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063ac25-dd7e-4a84-a225-cc72a60348eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.129317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063ac25-dd7e-4a84-a225-cc72a60348eb None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.129600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063ac25-dd7e-4a84-a225-cc72a60348eb None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.129891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063ac25-dd7e-4a84-a225-cc72a60348eb None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.130538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063ac25-dd7e-4a84-a225-cc72a60348eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:04.130841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063ac25-dd7e-4a84-a225-cc72a60348eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:04.132241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063ac25-dd7e-4a84-a225-cc72a60348eb None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:04.143476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 43/78] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:04.143609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c282358-01b2-4d71-8ab1-88af61f6235b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.144279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063ac25-dd7e-4a84-a225-cc72a60348eb None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.181841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436def25-5884-47b5-b4d9-084b7001cba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.182371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436def25-5884-47b5-b4d9-084b7001cba8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.182651 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436def25-5884-47b5-b4d9-084b7001cba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.182921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436def25-5884-47b5-b4d9-084b7001cba8 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.183599 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436def25-5884-47b5-b4d9-084b7001cba8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:04.183952 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436def25-5884-47b5-b4d9-084b7001cba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:04.185353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436def25-5884-47b5-b4d9-084b7001cba8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:04.191796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:04.193702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 36/79] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:04.194669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436def25-5884-47b5-b4d9-084b7001cba8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.247965 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063ac25-dd7e-4a84-a225-cc72a60348eb None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.260915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:04.283130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57e6b4-30e3-4755-b4d3-6eba23ae2bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.283759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57e6b4-30e3-4755-b4d3-6eba23ae2bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.284045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57e6b4-30e3-4755-b4d3-6eba23ae2bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.284334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57e6b4-30e3-4755-b4d3-6eba23ae2bc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.285450 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57e6b4-30e3-4755-b4d3-6eba23ae2bc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:04.286937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57e6b4-30e3-4755-b4d3-6eba23ae2bc0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:04.288519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 44/80] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:04.289313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57e6b4-30e3-4755-b4d3-6eba23ae2bc0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.297827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436def25-5884-47b5-b4d9-084b7001cba8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.333916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fac2d-f9a8-49de-8ece-5d3a98f23f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.334524 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fac2d-f9a8-49de-8ece-5d3a98f23f69 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.334800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fac2d-f9a8-49de-8ece-5d3a98f23f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.335071 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fac2d-f9a8-49de-8ece-5d3a98f23f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.389634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57e6b4-30e3-4755-b4d3-6eba23ae2bc0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.402936 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962fac2d-f9a8-49de-8ece-5d3a98f23f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:04.403778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:04.404452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962fac2d-f9a8-49de-8ece-5d3a98f23f69 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:04.426389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a660556-ca58-4c5c-a797-25e72dc7a40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.427003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a660556-ca58-4c5c-a797-25e72dc7a40e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.427286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a660556-ca58-4c5c-a797-25e72dc7a40e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.427561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a660556-ca58-4c5c-a797-25e72dc7a40e None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.428236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a660556-ca58-4c5c-a797-25e72dc7a40e None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:04.428539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a660556-ca58-4c5c-a797-25e72dc7a40e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:04.429978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a660556-ca58-4c5c-a797-25e72dc7a40e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:04.438377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 45/81] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:04.439120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a660556-ca58-4c5c-a797-25e72dc7a40e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.468965 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:04.505186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 37/82] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:04.506021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fac2d-f9a8-49de-8ece-5d3a98f23f69 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.544713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a660556-ca58-4c5c-a797-25e72dc7a40e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.579005 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b9f53-b211-4236-b3a9-4b8718588303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.579721 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b9f53-b211-4236-b3a9-4b8718588303 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.580065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b9f53-b211-4236-b3a9-4b8718588303 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.580405 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b9f53-b211-4236-b3a9-4b8718588303 None admin] PATH_INFO: `/v3/users/8563b95970a84781849db7018a225599` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.581614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b9f53-b211-4236-b3a9-4b8718588303 None admin] RBAC: Authorizing `identity:get_user(user_id=8563b95970a84781849db7018a225599)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:04.583078 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b9f53-b211-4236-b3a9-4b8718588303 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:04.584684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 46/83] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:04 2022] GET /identity/v3/users/8563b95970a84781849db7018a225599 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:04.585423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b9f53-b211-4236-b3a9-4b8718588303 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.609130 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fac2d-f9a8-49de-8ece-5d3a98f23f69 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.633048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:04.645170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a789a90-c5d2-47a0-a54e-b0c399ad79c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.645800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a789a90-c5d2-47a0-a54e-b0c399ad79c1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.646080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a789a90-c5d2-47a0-a54e-b0c399ad79c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.646353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a789a90-c5d2-47a0-a54e-b0c399ad79c1 None admin] PATH_INFO: `/v3/users/fe15d570e61b4d028bf4767532315b26` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.647489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a789a90-c5d2-47a0-a54e-b0c399ad79c1 None admin] RBAC: Authorizing `identity:get_user(user_id=fe15d570e61b4d028bf4767532315b26)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:04.649023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a789a90-c5d2-47a0-a54e-b0c399ad79c1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:04.650623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 38/84] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:04 2022] GET /identity/v3/users/fe15d570e61b4d028bf4767532315b26 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:04.651380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a789a90-c5d2-47a0-a54e-b0c399ad79c1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.685643 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:04.695314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b9f53-b211-4236-b3a9-4b8718588303 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.731823 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9c988-a02f-47b9-aaf3-8076218bf25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.732762 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9c988-a02f-47b9-aaf3-8076218bf25b None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.733053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9c988-a02f-47b9-aaf3-8076218bf25b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.733330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9c988-a02f-47b9-aaf3-8076218bf25b None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.733980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9c988-a02f-47b9-aaf3-8076218bf25b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:04.735444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9c988-a02f-47b9-aaf3-8076218bf25b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:04.754822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a789a90-c5d2-47a0-a54e-b0c399ad79c1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.756959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 47/85] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:04 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:04.757853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9c988-a02f-47b9-aaf3-8076218bf25b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.801609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fc173-ad70-44c9-86e8-cf948ce85baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.802363 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fc173-ad70-44c9-86e8-cf948ce85baa None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.802633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fc173-ad70-44c9-86e8-cf948ce85baa None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.802906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fc173-ad70-44c9-86e8-cf948ce85baa None admin] PATH_INFO: `/v3/users/8563b95970a84781849db7018a225599` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.804209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fc173-ad70-44c9-86e8-cf948ce85baa None admin] RBAC: Authorizing `identity:get_user(user_id=8563b95970a84781849db7018a225599)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:04.805762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fc173-ad70-44c9-86e8-cf948ce85baa None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:04.812599 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 39/86] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:04 2022] GET /identity/v3/users/8563b95970a84781849db7018a225599 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:04.812599 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fc173-ad70-44c9-86e8-cf948ce85baa None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.908097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:04.912028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9c988-a02f-47b9-aaf3-8076218bf25b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.958021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fc173-ad70-44c9-86e8-cf948ce85baa None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.959681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:04.961490 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3481f5-c281-45e4-9aa9-aeb835cdde5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.962238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3481f5-c281-45e4-9aa9-aeb835cdde5f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.962521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3481f5-c281-45e4-9aa9-aeb835cdde5f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.962798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3481f5-c281-45e4-9aa9-aeb835cdde5f None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.965107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3481f5-c281-45e4-9aa9-aeb835cdde5f None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:04.966235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3481f5-c281-45e4-9aa9-aeb835cdde5f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:04.968459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 48/87] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:04 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:04.968676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3481f5-c281-45e4-9aa9-aeb835cdde5f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:05.013329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b10a67-0699-4a75-a01f-685ff9bd368a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:05.013943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b10a67-0699-4a75-a01f-685ff9bd368a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:05.014220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b10a67-0699-4a75-a01f-685ff9bd368a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:05.014669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b10a67-0699-4a75-a01f-685ff9bd368a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:05.106661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b10a67-0699-4a75-a01f-685ff9bd368a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:05.108197 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3481f5-c281-45e4-9aa9-aeb835cdde5f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:05.108316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b10a67-0699-4a75-a01f-685ff9bd368a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:05.163507 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3450e94-4fcf-45a9-b2c9-c3d87976667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:05.164882 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3450e94-4fcf-45a9-b2c9-c3d87976667f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:05.165892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3450e94-4fcf-45a9-b2c9-c3d87976667f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:05.166105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3450e94-4fcf-45a9-b2c9-c3d87976667f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:05.225916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:05.237174 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 40/88] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:05.238286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b10a67-0699-4a75-a01f-685ff9bd368a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:05.243454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:05.287732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3450e94-4fcf-45a9-b2c9-c3d87976667f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:05.288642 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3450e94-4fcf-45a9-b2c9-c3d87976667f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:05.409909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b10a67-0699-4a75-a01f-685ff9bd368a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:05.451667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 49/89] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:05.452954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3450e94-4fcf-45a9-b2c9-c3d87976667f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:05.458473 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55124980-815c-4610-8b1f-2e62a3da66f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:05.459381 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55124980-815c-4610-8b1f-2e62a3da66f2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:05.459851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55124980-815c-4610-8b1f-2e62a3da66f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:05.460315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55124980-815c-4610-8b1f-2e62a3da66f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:05.561509 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:05.577005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55124980-815c-4610-8b1f-2e62a3da66f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:05.578864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55124980-815c-4610-8b1f-2e62a3da66f2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:05.581630 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:05.616547 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3450e94-4fcf-45a9-b2c9-c3d87976667f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:05.668382 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69487b-10f5-43b0-a676-22c76b999fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:05.669182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69487b-10f5-43b0-a676-22c76b999fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:05.669471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69487b-10f5-43b0-a676-22c76b999fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:05.669751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69487b-10f5-43b0-a676-22c76b999fb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:05.670500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69487b-10f5-43b0-a676-22c76b999fb5 None admin] RBAC: Adding query filter params (role.id=3647def0715f4fd4991f82b06815ee38, scope.project.id=79572ca8d9504396b675929feb75e9f2, user.id=8563b95970a84781849db7018a225599) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:05.670814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69487b-10f5-43b0-a676-22c76b999fb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:05.672293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69487b-10f5-43b0-a676-22c76b999fb5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:05.682901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 50/90] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:05 2022] GET /identity/v3/role_assignments?user.id=8563b95970a84781849db7018a225599&scope.project.id=79572ca8d9504396b675929feb75e9f2&role.id=3647def0715f4fd4991f82b06815ee38 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:05.683786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69487b-10f5-43b0-a676-22c76b999fb5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:05.769429 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 41/91] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:05.771037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55124980-815c-4610-8b1f-2e62a3da66f2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:05.830392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69487b-10f5-43b0-a676-22c76b999fb5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:05.871462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:05.894433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7f0e3-4c0b-4fff-8083-75cf4694a6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:05.895091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7f0e3-4c0b-4fff-8083-75cf4694a6ca None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:05.895373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7f0e3-4c0b-4fff-8083-75cf4694a6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:05.895648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7f0e3-4c0b-4fff-8083-75cf4694a6ca None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:05.896888 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7f0e3-4c0b-4fff-8083-75cf4694a6ca None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:05.898373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7f0e3-4c0b-4fff-8083-75cf4694a6ca None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:05.900482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 51/92] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:05 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:05.901300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7f0e3-4c0b-4fff-8083-75cf4694a6ca None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:05.971731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55124980-815c-4610-8b1f-2e62a3da66f2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:05.972980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:06.051336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ce04f-f805-4471-8cfd-a7001c57d14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:06.051992 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ce04f-f805-4471-8cfd-a7001c57d14b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.052270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ce04f-f805-4471-8cfd-a7001c57d14b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.052539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ce04f-f805-4471-8cfd-a7001c57d14b None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.053743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ce04f-f805-4471-8cfd-a7001c57d14b None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:06.055324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ce04f-f805-4471-8cfd-a7001c57d14b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:06.057546 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 42/93] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:05 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:06.058731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ce04f-f805-4471-8cfd-a7001c57d14b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:06.103039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7f0e3-4c0b-4fff-8083-75cf4694a6ca None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:06.178804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ce04f-f805-4471-8cfd-a7001c57d14b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:06.216587 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04a79a-868e-43ee-a151-47f415d7c0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:06.217316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04a79a-868e-43ee-a151-47f415d7c0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.217792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04a79a-868e-43ee-a151-47f415d7c0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.218069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04a79a-868e-43ee-a151-47f415d7c0e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.218294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb789019-fe74-434e-af16-cfe246263efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:06.218799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04a79a-868e-43ee-a151-47f415d7c0e6 None admin] RBAC: Adding query filter params (role.id=16317a22e9c94a8886f02efb75ab456e, scope.project.id=79572ca8d9504396b675929feb75e9f2, user.id=8563b95970a84781849db7018a225599) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:06.218979 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb789019-fe74-434e-af16-cfe246263efd None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.219144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04a79a-868e-43ee-a151-47f415d7c0e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:06.219231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb789019-fe74-434e-af16-cfe246263efd None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.219486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb789019-fe74-434e-af16-cfe246263efd None admin] PATH_INFO: `/v3/projects/f5e763514f7348d3913834f41a18728c` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.220690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04a79a-868e-43ee-a151-47f415d7c0e6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:06.225347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb789019-fe74-434e-af16-cfe246263efd None admin] RBAC: Authorizing `identity:get_project(project_id=f5e763514f7348d3913834f41a18728c)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:06.226761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 43/94] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:06 2022] GET /identity/v3/role_assignments?user.id=8563b95970a84781849db7018a225599&scope.project.id=79572ca8d9504396b675929feb75e9f2&role.id=16317a22e9c94a8886f02efb75ab456e => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:06.226867 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb789019-fe74-434e-af16-cfe246263efd None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:06.227577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04a79a-868e-43ee-a151-47f415d7c0e6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:06.228495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 52/95] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:05 2022] GET /identity/v3/projects/f5e763514f7348d3913834f41a18728c => generated 312 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:06.234750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb789019-fe74-434e-af16-cfe246263efd None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:06.289190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:06.337095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:06.384914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb789019-fe74-434e-af16-cfe246263efd None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:06.430339 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04a79a-868e-43ee-a151-47f415d7c0e6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:06.455939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03505f-bf4a-469b-b540-13f64831320c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:06.456623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03505f-bf4a-469b-b540-13f64831320c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.456934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03505f-bf4a-469b-b540-13f64831320c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.457210 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03505f-bf4a-469b-b540-13f64831320c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.457973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03505f-bf4a-469b-b540-13f64831320c None admin] RBAC: Adding query filter params (role.id=3647def0715f4fd4991f82b06815ee38, scope.project.id=f5e763514f7348d3913834f41a18728c, user.id=8563b95970a84781849db7018a225599) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:06.458286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03505f-bf4a-469b-b540-13f64831320c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:06.464681 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03505f-bf4a-469b-b540-13f64831320c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:06.473998 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 53/96] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:06 2022] GET /identity/v3/role_assignments?user.id=8563b95970a84781849db7018a225599&scope.project.id=f5e763514f7348d3913834f41a18728c&role.id=3647def0715f4fd4991f82b06815ee38 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:06.506704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f51428-fe5b-4a3c-9e8f-96af96da9ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:06.507323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f51428-fe5b-4a3c-9e8f-96af96da9ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.507602 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f51428-fe5b-4a3c-9e8f-96af96da9ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.507877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f51428-fe5b-4a3c-9e8f-96af96da9ba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.508614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f51428-fe5b-4a3c-9e8f-96af96da9ba8 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=79572ca8d9504396b675929feb75e9f2, user.id=fe15d570e61b4d028bf4767532315b26) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:06.510921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f51428-fe5b-4a3c-9e8f-96af96da9ba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:06.512532 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f51428-fe5b-4a3c-9e8f-96af96da9ba8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:06.519151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 44/97] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:06 2022] GET /identity/v3/role_assignments?user.id=fe15d570e61b4d028bf4767532315b26&scope.project.id=79572ca8d9504396b675929feb75e9f2&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:07.085771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f3cff-91c5-4ce3-bf99-f7289cbe77af None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:07.086129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f3cff-91c5-4ce3-bf99-f7289cbe77af None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:07.086340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f3cff-91c5-4ce3-bf99-f7289cbe77af None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:07.087705 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 54/98] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:07.095088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42693b0-c819-4e20-8e55-972245d8c2d9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:07.095562 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42693b0-c819-4e20-8e55-972245d8c2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:07.095631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42693b0-c819-4e20-8e55-972245d8c2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:07.120065 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b42693b0-c819-4e20-8e55-972245d8c2d9 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:07.120065 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b42693b0-c819-4e20-8e55-972245d8c2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:07.208202 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 45/99] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:07.267941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812272d1-9daf-4dd3-9632-800b6039648c None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:07.268426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812272d1-9daf-4dd3-9632-800b6039648c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:07.268426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812272d1-9daf-4dd3-9632-800b6039648c None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:07.269892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 55/100] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:07.286610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6d5bd-da82-407a-86a8-9b31cdbcf9eb None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:07.286906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6d5bd-da82-407a-86a8-9b31cdbcf9eb None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:07.287163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6d5bd-da82-407a-86a8-9b31cdbcf9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:07.314240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-bab6d5bd-da82-407a-86a8-9b31cdbcf9eb None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:07.315445 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bab6d5bd-da82-407a-86a8-9b31cdbcf9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:07.340948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:07.404569 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 46/101] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:07.761565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812272d1-9daf-4dd3-9632-800b6039648c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:07.820557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6d5bd-da82-407a-86a8-9b31cdbcf9eb None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:07.929588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812272d1-9daf-4dd3-9632-800b6039648c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:07.975663 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:07.986674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69014d-60a8-4ca2-92c6-c774dcf6172c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:07.987319 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69014d-60a8-4ca2-92c6-c774dcf6172c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:07.987597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69014d-60a8-4ca2-92c6-c774dcf6172c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:07.987865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69014d-60a8-4ca2-92c6-c774dcf6172c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:07.990988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6d5bd-da82-407a-86a8-9b31cdbcf9eb None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:08.006904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69014d-60a8-4ca2-92c6-c774dcf6172c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-de69014d-60a8-4ca2-92c6-c774dcf6172c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:08.011670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:08.012608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:08.013485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:08.013485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:08.013485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:08.013485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:08.013485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:08.013485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:08.013485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:08.013485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:08.013485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:08.013485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:08.013485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:08.013485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:08.021077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 56/102] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:08.028404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69014d-60a8-4ca2-92c6-c774dcf6172c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:08.081385 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6262dc-fb0b-404c-84c4-bb962ae95dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:08.082031 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6262dc-fb0b-404c-84c4-bb962ae95dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:08.082316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6262dc-fb0b-404c-84c4-bb962ae95dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:08.082592 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6262dc-fb0b-404c-84c4-bb962ae95dd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:08.083752 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6262dc-fb0b-404c-84c4-bb962ae95dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:08.085371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6262dc-fb0b-404c-84c4-bb962ae95dd0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:08.086969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 47/103] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:08.093505 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6262dc-fb0b-404c-84c4-bb962ae95dd0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:08.157011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:08.193984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69014d-60a8-4ca2-92c6-c774dcf6172c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:08.276018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6262dc-fb0b-404c-84c4-bb962ae95dd0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:08.309443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aafa98-c739-4f31-b79a-a0fa11fb6b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:08.310086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aafa98-c739-4f31-b79a-a0fa11fb6b98 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:08.310356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aafa98-c739-4f31-b79a-a0fa11fb6b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:08.310628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aafa98-c739-4f31-b79a-a0fa11fb6b98 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:08.311312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aafa98-c739-4f31-b79a-a0fa11fb6b98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:08.311617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aafa98-c739-4f31-b79a-a0fa11fb6b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:08.313056 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aafa98-c739-4f31-b79a-a0fa11fb6b98 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:08.327542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72073ea-f94e-4e20-b522-bd4665cb832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:08.328162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72073ea-f94e-4e20-b522-bd4665cb832c None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:08.328503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72073ea-f94e-4e20-b522-bd4665cb832c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:08.328820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72073ea-f94e-4e20-b522-bd4665cb832c None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:08.329542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72073ea-f94e-4e20-b522-bd4665cb832c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:08.331276 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72073ea-f94e-4e20-b522-bd4665cb832c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:08.333939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 57/104] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:08.339805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aafa98-c739-4f31-b79a-a0fa11fb6b98 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:08.386877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 48/105] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:19:08 2022] POST /identity/v3/projects => generated 302 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:08.429121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:08.483155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aafa98-c739-4f31-b79a-a0fa11fb6b98 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:08.524780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bc125-ae53-48e5-852d-b1391a3fd651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:08.525490 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bc125-ae53-48e5-852d-b1391a3fd651 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:08.525778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bc125-ae53-48e5-852d-b1391a3fd651 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:08.526108 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bc125-ae53-48e5-852d-b1391a3fd651 None admin] PATH_INFO: `/v3/users/49d9450bae4b4227a60bc3addec76c0e` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:08.544362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58bc125-ae53-48e5-852d-b1391a3fd651 None admin] RBAC: Authorizing `identity:get_user(user_id=49d9450bae4b4227a60bc3addec76c0e)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:08.546721 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58bc125-ae53-48e5-852d-b1391a3fd651 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:08.550117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 58/106] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:08 2022] GET /identity/v3/users/49d9450bae4b4227a60bc3addec76c0e => generated 289 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:08.556410 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72073ea-f94e-4e20-b522-bd4665cb832c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:08.664591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:08.691997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72073ea-f94e-4e20-b522-bd4665cb832c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:08.740356 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f90b39-c654-499b-9092-6694be24ca6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:08.741012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f90b39-c654-499b-9092-6694be24ca6f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:08.741297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f90b39-c654-499b-9092-6694be24ca6f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:08.741570 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f90b39-c654-499b-9092-6694be24ca6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:08.831026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f90b39-c654-499b-9092-6694be24ca6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:08.832564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f90b39-c654-499b-9092-6694be24ca6f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:08.913298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:08.936986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 49/107] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:08.941602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bc125-ae53-48e5-852d-b1391a3fd651 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:09.019265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712947ea-356c-4db0-804f-97cf7910711c None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.019609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712947ea-356c-4db0-804f-97cf7910711c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.019901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712947ea-356c-4db0-804f-97cf7910711c None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.021397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 50/108] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:09.026842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef23d4a-0a01-4fda-baf4-95a4db025f40 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.027127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef23d4a-0a01-4fda-baf4-95a4db025f40 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.027403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef23d4a-0a01-4fda-baf4-95a4db025f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.060978 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1ef23d4a-0a01-4fda-baf4-95a4db025f40 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:09.062268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1ef23d4a-0a01-4fda-baf4-95a4db025f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:09.100479 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bc125-ae53-48e5-852d-b1391a3fd651 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:09.145330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd980a90-ad6e-476e-853b-cbfcce253fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:09.147017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:09.148128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd980a90-ad6e-476e-853b-cbfcce253fbb None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.148128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd980a90-ad6e-476e-853b-cbfcce253fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.148373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd980a90-ad6e-476e-853b-cbfcce253fbb None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.149601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd980a90-ad6e-476e-853b-cbfcce253fbb None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:09.151149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd980a90-ad6e-476e-853b-cbfcce253fbb None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:09.153748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 59/109] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:08 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:09.154308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 51/110] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:09.154987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c255c3-1599-4d92-ba6f-33c2b573f4be None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.161048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c255c3-1599-4d92-ba6f-33c2b573f4be None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.161334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c255c3-1599-4d92-ba6f-33c2b573f4be None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.162921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 60/111] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:09.163886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef23d4a-0a01-4fda-baf4-95a4db025f40 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:09.182181 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee8e60-ec60-4b73-bcc3-8fa1e030d835 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.182488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee8e60-ec60-4b73-bcc3-8fa1e030d835 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.182744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee8e60-ec60-4b73-bcc3-8fa1e030d835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.222151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-98ee8e60-ec60-4b73-bcc3-8fa1e030d835 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:09.223387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-98ee8e60-ec60-4b73-bcc3-8fa1e030d835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:09.329767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 61/112] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:09.331178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8d023-3f84-40d4-9eb6-860be146b2c7 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.331497 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8d023-3f84-40d4-9eb6-860be146b2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.331778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8d023-3f84-40d4-9eb6-860be146b2c7 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.334704 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 62/113] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:09.340408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a0e95-c78e-4f4b-8879-8b09372d1cee None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.340719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a0e95-c78e-4f4b-8879-8b09372d1cee None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.340999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a0e95-c78e-4f4b-8879-8b09372d1cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.371271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef23d4a-0a01-4fda-baf4-95a4db025f40 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:09.377902 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-385a0e95-c78e-4f4b-8879-8b09372d1cee None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:09.379130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-385a0e95-c78e-4f4b-8879-8b09372d1cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:09.381694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:09.410775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68d2aa-109c-42e4-a3d8-e50aa1827756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:09.411683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68d2aa-109c-42e4-a3d8-e50aa1827756 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.412169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68d2aa-109c-42e4-a3d8-e50aa1827756 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.412616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68d2aa-109c-42e4-a3d8-e50aa1827756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.414219 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68d2aa-109c-42e4-a3d8-e50aa1827756 None admin] RBAC: Adding query filter params (role.id=26063cb085b6415989a19c8e7dc2c3c5, scope.project.id=79572ca8d9504396b675929feb75e9f2, user.id=49d9450bae4b4227a60bc3addec76c0e) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:09.414710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68d2aa-109c-42e4-a3d8-e50aa1827756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:09.416354 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68d2aa-109c-42e4-a3d8-e50aa1827756 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:09.423074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 52/114] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:09 2022] GET /identity/v3/role_assignments?user.id=49d9450bae4b4227a60bc3addec76c0e&scope.project.id=79572ca8d9504396b675929feb75e9f2&role.id=26063cb085b6415989a19c8e7dc2c3c5 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:09.500687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68d2aa-109c-42e4-a3d8-e50aa1827756 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:09.522501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 63/115] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:09.548589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ba30f-2c43-4ec3-8c71-fadab0e8a457 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.548959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ba30f-2c43-4ec3-8c71-fadab0e8a457 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.549240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ba30f-2c43-4ec3-8c71-fadab0e8a457 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.550820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 64/116] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:09.556338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ec30b-aafc-4ca5-944e-c5dfdcc94625 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.556614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ec30b-aafc-4ca5-944e-c5dfdcc94625 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.556975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ec30b-aafc-4ca5-944e-c5dfdcc94625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.586578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d19ec30b-aafc-4ca5-944e-c5dfdcc94625 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:09.587795 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d19ec30b-aafc-4ca5-944e-c5dfdcc94625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:09.665809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:09.699077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68d2aa-109c-42e4-a3d8-e50aa1827756 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:09.733566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 65/117] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:09.767580 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86fd54-7015-4a14-8f57-b5a1e5a217a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:09.768274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86fd54-7015-4a14-8f57-b5a1e5a217a8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.768559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86fd54-7015-4a14-8f57-b5a1e5a217a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.768859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86fd54-7015-4a14-8f57-b5a1e5a217a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.783316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86fd54-7015-4a14-8f57-b5a1e5a217a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-dd86fd54-7015-4a14-8f57-b5a1e5a217a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:09.788746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:09.789576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:09.790325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 53/118] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:09.796737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ec30b-aafc-4ca5-944e-c5dfdcc94625 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:09.812615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86fd54-7015-4a14-8f57-b5a1e5a217a8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:09.859153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:09.929980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ec30b-aafc-4ca5-944e-c5dfdcc94625 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:09.978045 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86fd54-7015-4a14-8f57-b5a1e5a217a8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.013427 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770184d-ac24-4daa-88e8-2c27960e5887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.014066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770184d-ac24-4daa-88e8-2c27960e5887 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.014412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770184d-ac24-4daa-88e8-2c27960e5887 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.014622 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770184d-ac24-4daa-88e8-2c27960e5887 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.015017 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d14c4e-8f98-4ba2-a70d-80c9f1cbd099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.015298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770184d-ac24-4daa-88e8-2c27960e5887 None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:10.015604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770184d-ac24-4daa-88e8-2c27960e5887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.015783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d14c4e-8f98-4ba2-a70d-80c9f1cbd099 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.015956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d14c4e-8f98-4ba2-a70d-80c9f1cbd099 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.016181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d14c4e-8f98-4ba2-a70d-80c9f1cbd099 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.017048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770184d-ac24-4daa-88e8-2c27960e5887 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.027543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d14c4e-8f98-4ba2-a70d-80c9f1cbd099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-25d14c4e-8f98-4ba2-a70d-80c9f1cbd099 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:10.033614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:10.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:10.035207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 54/119] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:10.035820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d14c4e-8f98-4ba2-a70d-80c9f1cbd099 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.046619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 66/120] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:10.047480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770184d-ac24-4daa-88e8-2c27960e5887 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.087536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.168642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.179128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d14c4e-8f98-4ba2-a70d-80c9f1cbd099 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.190028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770184d-ac24-4daa-88e8-2c27960e5887 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.215646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ebe6f-69da-4a1a-9a20-63d9cf3b95d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.216731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ebe6f-69da-4a1a-9a20-63d9cf3b95d5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.217229 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ebe6f-69da-4a1a-9a20-63d9cf3b95d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.217507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ebe6f-69da-4a1a-9a20-63d9cf3b95d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.234768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ebe6f-69da-4a1a-9a20-63d9cf3b95d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:10.239353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bd290-bbd5-4a52-b0cf-50ba9170f2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-ce8ebe6f-69da-4a1a-9a20-63d9cf3b95d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:10.239956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:10.240775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:10.241537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 55/121] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:10.242129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ebe6f-69da-4a1a-9a20-63d9cf3b95d5 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.243041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bd290-bbd5-4a52-b0cf-50ba9170f2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.243556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bd290-bbd5-4a52-b0cf-50ba9170f2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.244025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bd290-bbd5-4a52-b0cf-50ba9170f2c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.245208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bd290-bbd5-4a52-b0cf-50ba9170f2c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:10.246848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bd290-bbd5-4a52-b0cf-50ba9170f2c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.248492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bd290-bbd5-4a52-b0cf-50ba9170f2c3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.259797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 67/122] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:10.261252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bd290-bbd5-4a52-b0cf-50ba9170f2c3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.351889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ebe6f-69da-4a1a-9a20-63d9cf3b95d5 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.387999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.394802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237587a-7c5d-4dd5-bcd4-36f98db798d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.395431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237587a-7c5d-4dd5-bcd4-36f98db798d2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.395704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237587a-7c5d-4dd5-bcd4-36f98db798d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.395971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237587a-7c5d-4dd5-bcd4-36f98db798d2 None admin] PATH_INFO: `/v3/users/8563b95970a84781849db7018a225599` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.397784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237587a-7c5d-4dd5-bcd4-36f98db798d2 None admin] RBAC: Authorizing `identity:get_user(user_id=8563b95970a84781849db7018a225599)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.399270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237587a-7c5d-4dd5-bcd4-36f98db798d2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.401070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 56/123] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/users/8563b95970a84781849db7018a225599 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:10.401659 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237587a-7c5d-4dd5-bcd4-36f98db798d2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.407592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bd290-bbd5-4a52-b0cf-50ba9170f2c3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.429870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.456614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c29c28-3ddc-4a89-a160-acb6d320958c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.457909 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c29c28-3ddc-4a89-a160-acb6d320958c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.458457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c29c28-3ddc-4a89-a160-acb6d320958c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.459467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c29c28-3ddc-4a89-a160-acb6d320958c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.475068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c29c28-3ddc-4a89-a160-acb6d320958c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-c4c29c28-3ddc-4a89-a160-acb6d320958c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:10.482748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:10.483598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:10.484354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:10.484354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:10.484354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:10.484354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:10.484354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:10.484354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:10.484354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:10.484354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:10.484354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:10.484354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:10.484354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:10.484354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:10.492063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 68/124] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:10.492063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c29c28-3ddc-4a89-a160-acb6d320958c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.525147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237587a-7c5d-4dd5-bcd4-36f98db798d2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.562108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092b4d3-bd51-406b-9159-6ad329795845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.562459 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092b4d3-bd51-406b-9159-6ad329795845 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.562740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092b4d3-bd51-406b-9159-6ad329795845 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.563014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092b4d3-bd51-406b-9159-6ad329795845 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.563677 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092b4d3-bd51-406b-9159-6ad329795845 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:10.563978 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092b4d3-bd51-406b-9159-6ad329795845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.565417 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092b4d3-bd51-406b-9159-6ad329795845 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.575034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 57/125] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:10.575590 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092b4d3-bd51-406b-9159-6ad329795845 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.626648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c29c28-3ddc-4a89-a160-acb6d320958c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.667765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f79f3-be79-4684-a41e-c68b46e5ea53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.668886 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f79f3-be79-4684-a41e-c68b46e5ea53 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.669380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f79f3-be79-4684-a41e-c68b46e5ea53 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.669869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f79f3-be79-4684-a41e-c68b46e5ea53 None admin] PATH_INFO: `/v3/users/fe15d570e61b4d028bf4767532315b26` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.671311 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f79f3-be79-4684-a41e-c68b46e5ea53 None admin] RBAC: Authorizing `identity:get_user(user_id=fe15d570e61b4d028bf4767532315b26)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.673112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f79f3-be79-4684-a41e-c68b46e5ea53 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.674929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 69/126] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/users/fe15d570e61b4d028bf4767532315b26 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:10.675902 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f79f3-be79-4684-a41e-c68b46e5ea53 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.690637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.705616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.721088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092b4d3-bd51-406b-9159-6ad329795845 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.769166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415cd0e-61b2-45e3-ad32-9e1dac6447b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.770047 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415cd0e-61b2-45e3-ad32-9e1dac6447b8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.770329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415cd0e-61b2-45e3-ad32-9e1dac6447b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.770600 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415cd0e-61b2-45e3-ad32-9e1dac6447b8 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.772019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415cd0e-61b2-45e3-ad32-9e1dac6447b8 None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.774056 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415cd0e-61b2-45e3-ad32-9e1dac6447b8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.776241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 58/127] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:10.777461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415cd0e-61b2-45e3-ad32-9e1dac6447b8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.798622 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f79f3-be79-4684-a41e-c68b46e5ea53 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.851181 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cdf90-7ec0-4002-81fd-854bada42472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.851793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cdf90-7ec0-4002-81fd-854bada42472 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.852074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cdf90-7ec0-4002-81fd-854bada42472 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.852347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cdf90-7ec0-4002-81fd-854bada42472 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.853043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cdf90-7ec0-4002-81fd-854bada42472 None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:10.853346 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cdf90-7ec0-4002-81fd-854bada42472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.855633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cdf90-7ec0-4002-81fd-854bada42472 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.866248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 70/128] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:10.871036 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cdf90-7ec0-4002-81fd-854bada42472 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.897300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415cd0e-61b2-45e3-ad32-9e1dac6447b8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.942141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd4678-73d5-4044-822d-c39996b8bef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.942920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd4678-73d5-4044-822d-c39996b8bef2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.943302 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd4678-73d5-4044-822d-c39996b8bef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.943580 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd4678-73d5-4044-822d-c39996b8bef2 None admin] PATH_INFO: `/v3/users/8563b95970a84781849db7018a225599` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.944990 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd4678-73d5-4044-822d-c39996b8bef2 None admin] RBAC: Authorizing `identity:get_user(user_id=8563b95970a84781849db7018a225599)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.947117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd4678-73d5-4044-822d-c39996b8bef2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.949493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 59/129] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/users/8563b95970a84781849db7018a225599 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:10.950631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd4678-73d5-4044-822d-c39996b8bef2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.981694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.989849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:11.008105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cdf90-7ec0-4002-81fd-854bada42472 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.048201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4413220-5531-4f90-9b18-bf88bcf38479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.049266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4413220-5531-4f90-9b18-bf88bcf38479 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.049765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4413220-5531-4f90-9b18-bf88bcf38479 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.050231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4413220-5531-4f90-9b18-bf88bcf38479 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.051672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4413220-5531-4f90-9b18-bf88bcf38479 None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.053470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4413220-5531-4f90-9b18-bf88bcf38479 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.055630 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 71/130] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:11.056624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4413220-5531-4f90-9b18-bf88bcf38479 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:11.096179 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd4678-73d5-4044-822d-c39996b8bef2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.143289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924eb36c-09da-43bf-a905-642769a00f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.143956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924eb36c-09da-43bf-a905-642769a00f3c None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.144238 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924eb36c-09da-43bf-a905-642769a00f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.144513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924eb36c-09da-43bf-a905-642769a00f3c None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2/users/8563b95970a84781849db7018a225599/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.146533 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924eb36c-09da-43bf-a905-642769a00f3c None admin] RBAC: Authorizing `identity:create_grant(project_id=79572ca8d9504396b675929feb75e9f2, user_id=8563b95970a84781849db7018a225599, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.148693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924eb36c-09da-43bf-a905-642769a00f3c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.167472 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 60/131] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:10 2022] PUT /identity/v3/projects/79572ca8d9504396b675929feb75e9f2/users/8563b95970a84781849db7018a225599/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:11.168227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924eb36c-09da-43bf-a905-642769a00f3c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:11.183645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4413220-5531-4f90-9b18-bf88bcf38479 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.271064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46051c28-c864-4c07-8330-611654128b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.271679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46051c28-c864-4c07-8330-611654128b43 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.271962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46051c28-c864-4c07-8330-611654128b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.272231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46051c28-c864-4c07-8330-611654128b43 None admin] PATH_INFO: `/v3/users/8563b95970a84781849db7018a225599` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.273526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46051c28-c864-4c07-8330-611654128b43 None admin] RBAC: Authorizing `identity:get_user(user_id=8563b95970a84781849db7018a225599)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.276364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46051c28-c864-4c07-8330-611654128b43 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.280904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 72/132] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:11 2022] GET /identity/v3/users/8563b95970a84781849db7018a225599 => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:11.282521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46051c28-c864-4c07-8330-611654128b43 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:11.318220 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:11.323215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:11.360367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924eb36c-09da-43bf-a905-642769a00f3c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.405167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec1f86-26b3-45b9-b73b-882e64e8a526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.406150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec1f86-26b3-45b9-b73b-882e64e8a526 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.407090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec1f86-26b3-45b9-b73b-882e64e8a526 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.407567 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec1f86-26b3-45b9-b73b-882e64e8a526 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.409554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec1f86-26b3-45b9-b73b-882e64e8a526 None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.411924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec1f86-26b3-45b9-b73b-882e64e8a526 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.415164 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 61/133] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:11 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:11.416251 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec1f86-26b3-45b9-b73b-882e64e8a526 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:11.424464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46051c28-c864-4c07-8330-611654128b43 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.474567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e937681-4c6c-4a56-a550-fcde048a5b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.475654 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e937681-4c6c-4a56-a550-fcde048a5b33 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.476147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e937681-4c6c-4a56-a550-fcde048a5b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.476738 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e937681-4c6c-4a56-a550-fcde048a5b33 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2/users/fe15d570e61b4d028bf4767532315b26/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.478998 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e937681-4c6c-4a56-a550-fcde048a5b33 None admin] RBAC: Authorizing `identity:create_grant(project_id=79572ca8d9504396b675929feb75e9f2, user_id=fe15d570e61b4d028bf4767532315b26, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.480945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e937681-4c6c-4a56-a550-fcde048a5b33 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.500916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 73/134] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:11 2022] PUT /identity/v3/projects/79572ca8d9504396b675929feb75e9f2/users/fe15d570e61b4d028bf4767532315b26/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:11.502660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3e310-7545-4412-b1cd-6d276c09bd8f None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.503171 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3e310-7545-4412-b1cd-6d276c09bd8f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.503657 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3e310-7545-4412-b1cd-6d276c09bd8f None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.505477 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 74/135] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:11.506667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3e310-7545-4412-b1cd-6d276c09bd8f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:11.531936 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec1f86-26b3-45b9-b73b-882e64e8a526 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.568563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab34350-f1a0-43fa-ac23-a4cd4d497ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.569232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab34350-f1a0-43fa-ac23-a4cd4d497ecc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.569512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab34350-f1a0-43fa-ac23-a4cd4d497ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.569784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab34350-f1a0-43fa-ac23-a4cd4d497ecc None admin] PATH_INFO: `/v3/projects/f5e763514f7348d3913834f41a18728c` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.570922 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab34350-f1a0-43fa-ac23-a4cd4d497ecc None admin] RBAC: Authorizing `identity:get_project(project_id=f5e763514f7348d3913834f41a18728c)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.572393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab34350-f1a0-43fa-ac23-a4cd4d497ecc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.574065 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 62/136] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:11 2022] GET /identity/v3/projects/f5e763514f7348d3913834f41a18728c => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:11.575385 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdf190-d427-4530-b645-aa36de52818b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.575737 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdf190-d427-4530-b645-aa36de52818b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.576351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdf190-d427-4530-b645-aa36de52818b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.608637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-10fdf190-d427-4530-b645-aa36de52818b None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:11.609843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-10fdf190-d427-4530-b645-aa36de52818b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:11.610934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:11.621543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:11.639719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3e310-7545-4412-b1cd-6d276c09bd8f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.682240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3c9a6-8e63-410d-8a46-a602ab4d6c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.683433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3c9a6-8e63-410d-8a46-a602ab4d6c4d None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.683972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3c9a6-8e63-410d-8a46-a602ab4d6c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.684534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3c9a6-8e63-410d-8a46-a602ab4d6c4d None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2/users/8563b95970a84781849db7018a225599/roles/16317a22e9c94a8886f02efb75ab456e` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.688166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 63/137] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:11.689144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdf190-d427-4530-b645-aa36de52818b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:11.689868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3c9a6-8e63-410d-8a46-a602ab4d6c4d None admin] RBAC: Authorizing `identity:create_grant(project_id=79572ca8d9504396b675929feb75e9f2, user_id=8563b95970a84781849db7018a225599, role_id=16317a22e9c94a8886f02efb75ab456e)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.691953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3c9a6-8e63-410d-8a46-a602ab4d6c4d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.710034 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 75/138] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:11 2022] PUT /identity/v3/projects/79572ca8d9504396b675929feb75e9f2/users/8563b95970a84781849db7018a225599/roles/16317a22e9c94a8886f02efb75ab456e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:11.712033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c8954-fa67-4a44-a27c-f540d461cdde None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.712539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c8954-fa67-4a44-a27c-f540d461cdde None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.713148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c8954-fa67-4a44-a27c-f540d461cdde None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.715492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 76/139] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:11.721627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3da55c-4dda-4bcc-b41a-edb3cb5630b4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.722373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3da55c-4dda-4bcc-b41a-edb3cb5630b4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.722840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3da55c-4dda-4bcc-b41a-edb3cb5630b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.744171 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ec3da55c-4dda-4bcc-b41a-edb3cb5630b4 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:11.745770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ec3da55c-4dda-4bcc-b41a-edb3cb5630b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:11.804252 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdf190-d427-4530-b645-aa36de52818b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.829956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 77/140] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:11.843157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd6368-a887-4509-83f5-f96a3bbc6719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.843842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd6368-a887-4509-83f5-f96a3bbc6719 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.844126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd6368-a887-4509-83f5-f96a3bbc6719 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.844400 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd6368-a887-4509-83f5-f96a3bbc6719 None admin] PATH_INFO: `/v3/projects/f5e763514f7348d3913834f41a18728c/users/8563b95970a84781849db7018a225599/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.846474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd6368-a887-4509-83f5-f96a3bbc6719 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5e763514f7348d3913834f41a18728c, user_id=8563b95970a84781849db7018a225599, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.847973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd6368-a887-4509-83f5-f96a3bbc6719 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.868330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 64/141] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:11 2022] PUT /identity/v3/projects/f5e763514f7348d3913834f41a18728c/users/8563b95970a84781849db7018a225599/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:12.132904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3da55c-4dda-4bcc-b41a-edb3cb5630b4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:12.214340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:12.237313 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd6368-a887-4509-83f5-f96a3bbc6719 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:12.241503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:12.280734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3da55c-4dda-4bcc-b41a-edb3cb5630b4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:12.345362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa75bb2-864d-446f-a560-962d5e4adc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:12.346002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa75bb2-864d-446f-a560-962d5e4adc6d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:12.346279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa75bb2-864d-446f-a560-962d5e4adc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:12.346550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa75bb2-864d-446f-a560-962d5e4adc6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:12.349246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa75bb2-864d-446f-a560-962d5e4adc6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:12.350715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa75bb2-864d-446f-a560-962d5e4adc6d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:12.352496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 78/142] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:12.359838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa75bb2-864d-446f-a560-962d5e4adc6d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:12.413966 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd6368-a887-4509-83f5-f96a3bbc6719 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:12.477913 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ab979-abe2-4994-8ac0-effa77a092b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:12.478551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ab979-abe2-4994-8ac0-effa77a092b2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:12.478827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ab979-abe2-4994-8ac0-effa77a092b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:12.479099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ab979-abe2-4994-8ac0-effa77a092b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:12.499545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ab979-abe2-4994-8ac0-effa77a092b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c24ab979-abe2-4994-8ac0-effa77a092b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:12.504934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:12.505769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:12.506513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 65/143] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:12.512860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ab979-abe2-4994-8ac0-effa77a092b2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:12.553981 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa75bb2-864d-446f-a560-962d5e4adc6d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:12.615138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:12.619136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816343bc-db92-4a38-b7a5-717608c3e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:12.619767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816343bc-db92-4a38-b7a5-717608c3e911 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:12.620059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816343bc-db92-4a38-b7a5-717608c3e911 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:12.620317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816343bc-db92-4a38-b7a5-717608c3e911 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:12.621002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816343bc-db92-4a38-b7a5-717608c3e911 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:12.622787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816343bc-db92-4a38-b7a5-717608c3e911 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:12.645386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:12.663722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ab979-abe2-4994-8ac0-effa77a092b2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:12.677593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 79/144] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:12 2022] POST /identity/v3/users => generated 283 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:12.717583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2065b2-7e84-4199-9139-2b5504d47371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:12.718444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2065b2-7e84-4199-9139-2b5504d47371 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:12.718444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2065b2-7e84-4199-9139-2b5504d47371 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:12.718708 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2065b2-7e84-4199-9139-2b5504d47371 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:12.719268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2065b2-7e84-4199-9139-2b5504d47371 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:12.719577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2065b2-7e84-4199-9139-2b5504d47371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:12.721014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2065b2-7e84-4199-9139-2b5504d47371 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:12.735377 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 66/145] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:12.741149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816343bc-db92-4a38-b7a5-717608c3e911 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:12.887730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816343bc-db92-4a38-b7a5-717608c3e911 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:12.922706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3b4b3-12ba-4115-8aab-976155b3c909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:12.923334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3b4b3-12ba-4115-8aab-976155b3c909 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:12.923609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3b4b3-12ba-4115-8aab-976155b3c909 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:12.923877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3b4b3-12ba-4115-8aab-976155b3c909 None admin] PATH_INFO: `/v3/users/49d9450bae4b4227a60bc3addec76c0e` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:12.925862 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3b4b3-12ba-4115-8aab-976155b3c909 None admin] RBAC: Authorizing `identity:get_user(user_id=49d9450bae4b4227a60bc3addec76c0e)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:12.927327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3b4b3-12ba-4115-8aab-976155b3c909 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:12.928989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 80/146] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:12 2022] GET /identity/v3/users/49d9450bae4b4227a60bc3addec76c0e => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:12.934692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2065b2-7e84-4199-9139-2b5504d47371 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:13.041504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2065b2-7e84-4199-9139-2b5504d47371 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:13.073340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:13.077310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f77a3-3052-4be9-b48d-f6f96c9ae4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:13.078090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f77a3-3052-4be9-b48d-f6f96c9ae4cc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:13.078641 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f77a3-3052-4be9-b48d-f6f96c9ae4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:13.078917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f77a3-3052-4be9-b48d-f6f96c9ae4cc None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:13.080124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f77a3-3052-4be9-b48d-f6f96c9ae4cc None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:13.081733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f77a3-3052-4be9-b48d-f6f96c9ae4cc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:13.085417 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 67/147] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:12 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:13.089935 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3b4b3-12ba-4115-8aab-976155b3c909 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:13.164073 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:13.221397 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3b4b3-12ba-4115-8aab-976155b3c909 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:13.270592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1a8df-d6df-43d7-bb4b-225a380ffe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:13.271654 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1a8df-d6df-43d7-bb4b-225a380ffe54 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:13.272144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1a8df-d6df-43d7-bb4b-225a380ffe54 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:13.272601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1a8df-d6df-43d7-bb4b-225a380ffe54 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2/users/49d9450bae4b4227a60bc3addec76c0e/roles/26063cb085b6415989a19c8e7dc2c3c5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:13.274747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1a8df-d6df-43d7-bb4b-225a380ffe54 None admin] RBAC: Authorizing `identity:create_grant(project_id=79572ca8d9504396b675929feb75e9f2, user_id=49d9450bae4b4227a60bc3addec76c0e, role_id=26063cb085b6415989a19c8e7dc2c3c5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:13.276405 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1a8df-d6df-43d7-bb4b-225a380ffe54 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:13.307982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 81/148] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:13 2022] PUT /identity/v3/projects/79572ca8d9504396b675929feb75e9f2/users/49d9450bae4b4227a60bc3addec76c0e/roles/26063cb085b6415989a19c8e7dc2c3c5 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:19:13.808248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbf454-e4c4-498b-aec4-586b64779273 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:13.808844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbf454-e4c4-498b-aec4-586b64779273 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:13.809321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbf454-e4c4-498b-aec4-586b64779273 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:13.810692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 68/149] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:13.816274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99692d-be0b-40bb-a754-c0b84d4a16e9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:13.816747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99692d-be0b-40bb-a754-c0b84d4a16e9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:13.817011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99692d-be0b-40bb-a754-c0b84d4a16e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:13.838082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-bd99692d-be0b-40bb-a754-c0b84d4a16e9 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:13.839585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bd99692d-be0b-40bb-a754-c0b84d4a16e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:13.930152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 82/150] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:14.447483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbaeee-00be-4c7a-9e5c-299b811a75fe None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.447855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbaeee-00be-4c7a-9e5c-299b811a75fe None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.448044 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbaeee-00be-4c7a-9e5c-299b811a75fe None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.449734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 69/151] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:14.457250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320ebb7-4440-409b-9fc4-66c7b3939ac0 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.457522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320ebb7-4440-409b-9fc4-66c7b3939ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.457793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320ebb7-4440-409b-9fc4-66c7b3939ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.482390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c320ebb7-4440-409b-9fc4-66c7b3939ac0 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:14.483587 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c320ebb7-4440-409b-9fc4-66c7b3939ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:14.490458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbaeee-00be-4c7a-9e5c-299b811a75fe None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:14.562605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:14.611073 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 83/152] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:14.644680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbaeee-00be-4c7a-9e5c-299b811a75fe None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:14.688252 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6480577-9507-490c-a724-ace3f9c50dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:14.688913 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6480577-9507-490c-a724-ace3f9c50dcb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.689188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6480577-9507-490c-a724-ace3f9c50dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.689454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6480577-9507-490c-a724-ace3f9c50dcb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.691593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0abcd8-9e55-4401-86c2-dc05e0c8b154 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.691881 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0abcd8-9e55-4401-86c2-dc05e0c8b154 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.692170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0abcd8-9e55-4401-86c2-dc05e0c8b154 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.693556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 84/153] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:14.702087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d9edd-24b8-485d-a5b5-334f6508f199 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.702327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d9edd-24b8-485d-a5b5-334f6508f199 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.702570 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d9edd-24b8-485d-a5b5-334f6508f199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.706162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6480577-9507-490c-a724-ace3f9c50dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d6480577-9507-490c-a724-ace3f9c50dcb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:14.711329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:14.712169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:14.712960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 70/154] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:14.719818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6480577-9507-490c-a724-ace3f9c50dcb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:14.743435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7e2d9edd-24b8-485d-a5b5-334f6508f199 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:14.744698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7e2d9edd-24b8-485d-a5b5-334f6508f199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:14.811336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:14.874585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 85/155] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:14.901322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6480577-9507-490c-a724-ace3f9c50dcb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:14.954935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38502408-c248-4b2d-bbc5-d822d849aa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:14.955457 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38502408-c248-4b2d-bbc5-d822d849aa71 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.955736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38502408-c248-4b2d-bbc5-d822d849aa71 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.956005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38502408-c248-4b2d-bbc5-d822d849aa71 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.956688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38502408-c248-4b2d-bbc5-d822d849aa71 None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:14.956997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38502408-c248-4b2d-bbc5-d822d849aa71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:14.958397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38502408-c248-4b2d-bbc5-d822d849aa71 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:14.973314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 71/156] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:14.980201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d9edd-24b8-485d-a5b5-334f6508f199 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:15.002475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673fe96-9ab3-4bb0-817e-7b84bf49c885 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.002790 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673fe96-9ab3-4bb0-817e-7b84bf49c885 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.003036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673fe96-9ab3-4bb0-817e-7b84bf49c885 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.004535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 72/157] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:15.009579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ec851-7dae-45a2-be3c-a75a6190ab73 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.009861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ec851-7dae-45a2-be3c-a75a6190ab73 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.010141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ec851-7dae-45a2-be3c-a75a6190ab73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.049709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c98ec851-7dae-45a2-be3c-a75a6190ab73 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:15.051025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c98ec851-7dae-45a2-be3c-a75a6190ab73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:15.111313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d9edd-24b8-485d-a5b5-334f6508f199 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:15.145190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:15.207098 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d450e9-ad51-4f32-bca3-ee4e84dee27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:15.207699 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d450e9-ad51-4f32-bca3-ee4e84dee27a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.207977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d450e9-ad51-4f32-bca3-ee4e84dee27a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.208245 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d450e9-ad51-4f32-bca3-ee4e84dee27a None admin] PATH_INFO: `/v3/users/8563b95970a84781849db7018a225599` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.213399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d450e9-ad51-4f32-bca3-ee4e84dee27a None admin] RBAC: Authorizing `identity:get_user(user_id=8563b95970a84781849db7018a225599)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:15.215359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d450e9-ad51-4f32-bca3-ee4e84dee27a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:15.217701 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 86/158] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:14 2022] GET /identity/v3/users/8563b95970a84781849db7018a225599 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:15.218733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d450e9-ad51-4f32-bca3-ee4e84dee27a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:15.219331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 73/159] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:15.222884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ec851-7dae-45a2-be3c-a75a6190ab73 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:15.313237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:15.416906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ec851-7dae-45a2-be3c-a75a6190ab73 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:15.442925 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d450e9-ad51-4f32-bca3-ee4e84dee27a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:15.507157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233219-3ed1-4ca5-8c20-7f5b53e7c5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:15.507157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233219-3ed1-4ca5-8c20-7f5b53e7c5ff None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.507157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233219-3ed1-4ca5-8c20-7f5b53e7c5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.507157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233219-3ed1-4ca5-8c20-7f5b53e7c5ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.520914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac50ee8-a25e-4f3b-80fa-f3846cecff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:15.521915 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac50ee8-a25e-4f3b-80fa-f3846cecff65 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.522416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac50ee8-a25e-4f3b-80fa-f3846cecff65 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.522878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac50ee8-a25e-4f3b-80fa-f3846cecff65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.540675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac50ee8-a25e-4f3b-80fa-f3846cecff65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-cac50ee8-a25e-4f3b-80fa-f3846cecff65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:15.546374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:15.547341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:15.548101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:15.548101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:15.548101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:15.548101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:15.548101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:15.548101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:15.548101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:15.548101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:15.548101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:15.548101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:15.548101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:15.548101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:15.550528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 87/160] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:15.551571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac50ee8-a25e-4f3b-80fa-f3846cecff65 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:15.637280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:15.641550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86233219-3ed1-4ca5-8c20-7f5b53e7c5ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:15.643043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86233219-3ed1-4ca5-8c20-7f5b53e7c5ff None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:15.709534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac50ee8-a25e-4f3b-80fa-f3846cecff65 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:15.725110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:15.765689 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f343b-2412-4db1-84b6-daf44d43181c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:15.766710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f343b-2412-4db1-84b6-daf44d43181c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.767232 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f343b-2412-4db1-84b6-daf44d43181c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.767719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f343b-2412-4db1-84b6-daf44d43181c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.784854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f343b-2412-4db1-84b6-daf44d43181c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3d0f343b-2412-4db1-84b6-daf44d43181c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:15.790503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:15.791399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:15.792168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:15.792168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:15.792168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:15.792168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:15.792168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:15.792168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:15.792168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:15.792168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:15.792168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:15.792168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:15.792168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:15.792168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:15.794650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 88/161] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:15.801784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f343b-2412-4db1-84b6-daf44d43181c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:15.804041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 74/162] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:19:15.804904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233219-3ed1-4ca5-8c20-7f5b53e7c5ff None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:16.006438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233219-3ed1-4ca5-8c20-7f5b53e7c5ff None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:16.019621 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:16.023375 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f343b-2412-4db1-84b6-daf44d43181c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:16.083512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8564d0e-5220-4d62-b116-f75ee23efb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:16.084254 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8564d0e-5220-4d62-b116-f75ee23efb70 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.084536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8564d0e-5220-4d62-b116-f75ee23efb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.084828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8564d0e-5220-4d62-b116-f75ee23efb70 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.085500 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8564d0e-5220-4d62-b116-f75ee23efb70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:16.085796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8564d0e-5220-4d62-b116-f75ee23efb70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:16.087424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8564d0e-5220-4d62-b116-f75ee23efb70 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:16.097443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afa0df-0376-456b-8ffa-3f70822c0cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:16.098397 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afa0df-0376-456b-8ffa-3f70822c0cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.098884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afa0df-0376-456b-8ffa-3f70822c0cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.099362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afa0df-0376-456b-8ffa-3f70822c0cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.100271 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afa0df-0376-456b-8ffa-3f70822c0cb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:16.100812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afa0df-0376-456b-8ffa-3f70822c0cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:16.102413 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afa0df-0376-456b-8ffa-3f70822c0cb1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:16.110514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 75/163] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:16.110514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8564d0e-5220-4d62-b116-f75ee23efb70 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:16.120915 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 89/164] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:16.122472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afa0df-0376-456b-8ffa-3f70822c0cb1 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:16.129346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:16.300564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8564d0e-5220-4d62-b116-f75ee23efb70 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:16.354228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afa0df-0376-456b-8ffa-3f70822c0cb1 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:16.364204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804b8dd-b1db-4a86-8731-d74c999138ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:16.364881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804b8dd-b1db-4a86-8731-d74c999138ba None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.365160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804b8dd-b1db-4a86-8731-d74c999138ba None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.365433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804b8dd-b1db-4a86-8731-d74c999138ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.394081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804b8dd-b1db-4a86-8731-d74c999138ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-5804b8dd-b1db-4a86-8731-d74c999138ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:16.399444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:16.400568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:16.401427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:16.401427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:16.401427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:16.401427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:16.401427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:16.401427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:16.401427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:16.401427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:16.401427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:16.401427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:16.401427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:16.401427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:16.408449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d9e4c-ea36-4e2a-9747-8cc3405a0a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:16.408830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 76/165] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:16.409739 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804b8dd-b1db-4a86-8731-d74c999138ba None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:16.413273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d9e4c-ea36-4e2a-9747-8cc3405a0a88 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.413563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d9e4c-ea36-4e2a-9747-8cc3405a0a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.413842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d9e4c-ea36-4e2a-9747-8cc3405a0a88 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.415015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d9e4c-ea36-4e2a-9747-8cc3405a0a88 None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:16.416556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d9e4c-ea36-4e2a-9747-8cc3405a0a88 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:16.418470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 90/166] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:16 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:16.419222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d9e4c-ea36-4e2a-9747-8cc3405a0a88 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:16.477110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:16.570059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d9e4c-ea36-4e2a-9747-8cc3405a0a88 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:16.628342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeedec4-72ac-4aca-bdea-07517dde8b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:16.629021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeedec4-72ac-4aca-bdea-07517dde8b0c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.629303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeedec4-72ac-4aca-bdea-07517dde8b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.629579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeedec4-72ac-4aca-bdea-07517dde8b0c None admin] PATH_INFO: `/v3/users/8563b95970a84781849db7018a225599` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.630728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeedec4-72ac-4aca-bdea-07517dde8b0c None admin] RBAC: Authorizing `identity:get_user(user_id=8563b95970a84781849db7018a225599)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:16.632221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeedec4-72ac-4aca-bdea-07517dde8b0c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:16.643543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 91/167] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:16 2022] GET /identity/v3/users/8563b95970a84781849db7018a225599 => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:16.644374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeedec4-72ac-4aca-bdea-07517dde8b0c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:16.659706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:16.749229 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804b8dd-b1db-4a86-8731-d74c999138ba None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:16.798216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:16.799349 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22336eb0-2397-4247-ab1a-50b6dd18d031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:16.800348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22336eb0-2397-4247-ab1a-50b6dd18d031 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.800916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22336eb0-2397-4247-ab1a-50b6dd18d031 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.801420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22336eb0-2397-4247-ab1a-50b6dd18d031 None admin] PATH_INFO: `/v3/users/fe15d570e61b4d028bf4767532315b26` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.803664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22336eb0-2397-4247-ab1a-50b6dd18d031 None admin] RBAC: Authorizing `identity:get_user(user_id=fe15d570e61b4d028bf4767532315b26)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:16.805389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22336eb0-2397-4247-ab1a-50b6dd18d031 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:16.806724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeedec4-72ac-4aca-bdea-07517dde8b0c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:16.807432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 77/168] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:16 2022] GET /identity/v3/users/fe15d570e61b4d028bf4767532315b26 => generated 247 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:16.808500 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22336eb0-2397-4247-ab1a-50b6dd18d031 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:16.879348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a80a8-3585-4ac7-abe6-0a3054459e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:16.879954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a80a8-3585-4ac7-abe6-0a3054459e84 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.880230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a80a8-3585-4ac7-abe6-0a3054459e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.880498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a80a8-3585-4ac7-abe6-0a3054459e84 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.881235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a80a8-3585-4ac7-abe6-0a3054459e84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:16.881537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a80a8-3585-4ac7-abe6-0a3054459e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:16.882933 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a80a8-3585-4ac7-abe6-0a3054459e84 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:16.897550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 92/169] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:16.899263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a80a8-3585-4ac7-abe6-0a3054459e84 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:16.959125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22336eb0-2397-4247-ab1a-50b6dd18d031 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.012080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666e044-1b4e-4875-8260-1d4e427acada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.013374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666e044-1b4e-4875-8260-1d4e427acada None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.013661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666e044-1b4e-4875-8260-1d4e427acada None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.013934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666e044-1b4e-4875-8260-1d4e427acada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.014679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666e044-1b4e-4875-8260-1d4e427acada None admin] RBAC: Adding query filter params (role.id=3647def0715f4fd4991f82b06815ee38, scope.project.id=79572ca8d9504396b675929feb75e9f2, user.id=8563b95970a84781849db7018a225599) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:17.014988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666e044-1b4e-4875-8260-1d4e427acada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:17.016453 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666e044-1b4e-4875-8260-1d4e427acada None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:17.020364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:17.032908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 78/170] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:16 2022] GET /identity/v3/role_assignments?user.id=8563b95970a84781849db7018a225599&scope.project.id=79572ca8d9504396b675929feb75e9f2&role.id=3647def0715f4fd4991f82b06815ee38 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:17.032908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574a8f0-706f-47c5-8d93-807f6b557f6e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.032908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574a8f0-706f-47c5-8d93-807f6b557f6e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.032908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574a8f0-706f-47c5-8d93-807f6b557f6e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.032908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 79/171] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:17.032908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574a8f0-706f-47c5-8d93-807f6b557f6e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:17.075077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a80a8-3585-4ac7-abe6-0a3054459e84 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.150959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e195d-5bdf-4abd-9436-a29a9e2b151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.153264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e195d-5bdf-4abd-9436-a29a9e2b151a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.154963 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e195d-5bdf-4abd-9436-a29a9e2b151a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.155985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e195d-5bdf-4abd-9436-a29a9e2b151a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.222139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:17.237444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574a8f0-706f-47c5-8d93-807f6b557f6e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.267875 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e195d-5bdf-4abd-9436-a29a9e2b151a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:17.269407 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e195d-5bdf-4abd-9436-a29a9e2b151a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:17.288908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70545b6-32b8-48a1-9814-3e915f21ec3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.289909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70545b6-32b8-48a1-9814-3e915f21ec3e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.290427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70545b6-32b8-48a1-9814-3e915f21ec3e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.290968 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70545b6-32b8-48a1-9814-3e915f21ec3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.369147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:17.371825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70545b6-32b8-48a1-9814-3e915f21ec3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:17.373739 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70545b6-32b8-48a1-9814-3e915f21ec3e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:17.387146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 93/172] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:17.388214 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e195d-5bdf-4abd-9436-a29a9e2b151a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:17.451123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:17.486936 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 80/173] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:17.488198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b3211-eec2-4aab-8233-95d9afc182fa None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.488475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b3211-eec2-4aab-8233-95d9afc182fa None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.488773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b3211-eec2-4aab-8233-95d9afc182fa None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.490002 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 81/174] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:17.490980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ff581-29c6-41d4-a657-9bcb887ce5f3 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.491251 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ff581-29c6-41d4-a657-9bcb887ce5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.491715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ff581-29c6-41d4-a657-9bcb887ce5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.502510 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e195d-5bdf-4abd-9436-a29a9e2b151a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.513166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-858ff581-29c6-41d4-a657-9bcb887ce5f3 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:17.514709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-858ff581-29c6-41d4-a657-9bcb887ce5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:17.536182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271daae-34bf-43e0-a109-076e2d4a6f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.536848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271daae-34bf-43e0-a109-076e2d4a6f4b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.537119 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271daae-34bf-43e0-a109-076e2d4a6f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.537382 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271daae-34bf-43e0-a109-076e2d4a6f4b None admin] PATH_INFO: `/v3/users/8563b95970a84781849db7018a225599` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.538479 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271daae-34bf-43e0-a109-076e2d4a6f4b None admin] RBAC: Authorizing `identity:get_user(user_id=8563b95970a84781849db7018a225599)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:17.539931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271daae-34bf-43e0-a109-076e2d4a6f4b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:17.541542 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 94/175] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:17 2022] GET /identity/v3/users/8563b95970a84781849db7018a225599 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:17.542613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771ce8e-a159-44ee-9cca-a334125461db None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.542885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771ce8e-a159-44ee-9cca-a334125461db None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.543225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771ce8e-a159-44ee-9cca-a334125461db None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.544677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 95/176] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:17.548826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771ce8e-a159-44ee-9cca-a334125461db None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:17.587643 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 82/177] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:17.588490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ff581-29c6-41d4-a657-9bcb887ce5f3 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:17.619154 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:17.659148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771ce8e-a159-44ee-9cca-a334125461db None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.687733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:17.691622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ff581-29c6-41d4-a657-9bcb887ce5f3 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.696922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d3aa7-416f-4e0b-b674-aa95ae9b7dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.697620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d3aa7-416f-4e0b-b674-aa95ae9b7dfd None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.697963 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d3aa7-416f-4e0b-b674-aa95ae9b7dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.698300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d3aa7-416f-4e0b-b674-aa95ae9b7dfd None admin] PATH_INFO: `/v3/projects/f5e763514f7348d3913834f41a18728c` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.699947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d3aa7-416f-4e0b-b674-aa95ae9b7dfd None admin] RBAC: Authorizing `identity:get_project(project_id=f5e763514f7348d3913834f41a18728c)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:17.702033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d3aa7-416f-4e0b-b674-aa95ae9b7dfd None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:17.703957 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 96/178] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:17 2022] GET /identity/v3/projects/f5e763514f7348d3913834f41a18728c => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:17.705095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff9a5c-cbcd-49eb-a038-0941df5510a3 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.705377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff9a5c-cbcd-49eb-a038-0941df5510a3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.705648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff9a5c-cbcd-49eb-a038-0941df5510a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.724004 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-edff9a5c-cbcd-49eb-a038-0941df5510a3 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:17.725237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-edff9a5c-cbcd-49eb-a038-0941df5510a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:17.740115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540275a-d78b-4846-bb78-e5f9047dc0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.741253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540275a-d78b-4846-bb78-e5f9047dc0fe None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.741669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540275a-d78b-4846-bb78-e5f9047dc0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.742320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540275a-d78b-4846-bb78-e5f9047dc0fe None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.743910 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540275a-d78b-4846-bb78-e5f9047dc0fe None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:17.745562 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540275a-d78b-4846-bb78-e5f9047dc0fe None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:17.748167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 83/179] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:17 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:17.750288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1ab11-3b47-4396-a814-303f7d88c4d6 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.750828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1ab11-3b47-4396-a814-303f7d88c4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.751344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1ab11-3b47-4396-a814-303f7d88c4d6 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.753601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 84/180] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:17.754700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1ab11-3b47-4396-a814-303f7d88c4d6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:17.798785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 97/181] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:17.802859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c27869-8626-45ba-b275-e92db28e472c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.804349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c27869-8626-45ba-b275-e92db28e472c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.805120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c27869-8626-45ba-b275-e92db28e472c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.846566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-55c27869-8626-45ba-b275-e92db28e472c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:17.850250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-55c27869-8626-45ba-b275-e92db28e472c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:17.868267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:17.874307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1ab11-3b47-4396-a814-303f7d88c4d6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.913802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3cb7c-e1b4-4a78-8eaa-21aab05a6eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.914450 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3cb7c-e1b4-4a78-8eaa-21aab05a6eba None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.914727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3cb7c-e1b4-4a78-8eaa-21aab05a6eba None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.915090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3cb7c-e1b4-4a78-8eaa-21aab05a6eba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.937321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 98/182] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:17.938516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c27869-8626-45ba-b275-e92db28e472c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:17.991856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3cb7c-e1b4-4a78-8eaa-21aab05a6eba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:17.993689 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3cb7c-e1b4-4a78-8eaa-21aab05a6eba None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:18.021687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:18.068099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c27869-8626-45ba-b275-e92db28e472c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.104406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aaa461-7725-49ee-b693-fb42e91176ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.105876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aaa461-7725-49ee-b693-fb42e91176ba None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.106379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aaa461-7725-49ee-b693-fb42e91176ba None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.106886 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aaa461-7725-49ee-b693-fb42e91176ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.107822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aaa461-7725-49ee-b693-fb42e91176ba None admin] RBAC: Adding query filter params (role.id=3647def0715f4fd4991f82b06815ee38, scope.project.id=f5e763514f7348d3913834f41a18728c, user.id=8563b95970a84781849db7018a225599) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:18.108759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aaa461-7725-49ee-b693-fb42e91176ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:18.110253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aaa461-7725-49ee-b693-fb42e91176ba None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:18.116976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 99/183] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:17 2022] GET /identity/v3/role_assignments?user.id=8563b95970a84781849db7018a225599&scope.project.id=f5e763514f7348d3913834f41a18728c&role.id=3647def0715f4fd4991f82b06815ee38 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:18.120732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aaa461-7725-49ee-b693-fb42e91176ba None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.130173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:18.136201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 85/184] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:18.138816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376b269-2942-428b-9c5b-d4ead0fae251 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.139279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376b269-2942-428b-9c5b-d4ead0fae251 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.139722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376b269-2942-428b-9c5b-d4ead0fae251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.163744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1376b269-2942-428b-9c5b-d4ead0fae251 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:18.165437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1376b269-2942-428b-9c5b-d4ead0fae251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:18.237051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aaa461-7725-49ee-b693-fb42e91176ba None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.250244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 86/185] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:18.251438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376b269-2942-428b-9c5b-d4ead0fae251 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.280871 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:18.286657 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55f95e-55e9-48e8-adcd-7a4ddff737d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.288112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f95e-55e9-48e8-adcd-7a4ddff737d9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.288567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f95e-55e9-48e8-adcd-7a4ddff737d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.288874 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f95e-55e9-48e8-adcd-7a4ddff737d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.289620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55f95e-55e9-48e8-adcd-7a4ddff737d9 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=79572ca8d9504396b675929feb75e9f2, user.id=fe15d570e61b4d028bf4767532315b26) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:18.289930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55f95e-55e9-48e8-adcd-7a4ddff737d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:18.291406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55f95e-55e9-48e8-adcd-7a4ddff737d9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:18.297777 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 100/186] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/role_assignments?user.id=fe15d570e61b4d028bf4767532315b26&scope.project.id=79572ca8d9504396b675929feb75e9f2&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:18.298589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55f95e-55e9-48e8-adcd-7a4ddff737d9 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.394422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376b269-2942-428b-9c5b-d4ead0fae251 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.427916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:18.438275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0848243-339f-4009-b7d0-a7ed84949641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.438931 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0848243-339f-4009-b7d0-a7ed84949641 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.439207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0848243-339f-4009-b7d0-a7ed84949641 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.439475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0848243-339f-4009-b7d0-a7ed84949641 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.451841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0848243-339f-4009-b7d0-a7ed84949641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:18.456592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55f95e-55e9-48e8-adcd-7a4ddff737d9 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e0848243-339f-4009-b7d0-a7ed84949641 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:18.456802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:18.457593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:18.458340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 87/187] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:18.458978 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0848243-339f-4009-b7d0-a7ed84949641 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.508855 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6f3a4-c48d-4364-93f3-d2c61ca4da15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.510224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6f3a4-c48d-4364-93f3-d2c61ca4da15 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.510727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6f3a4-c48d-4364-93f3-d2c61ca4da15 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.512229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6f3a4-c48d-4364-93f3-d2c61ca4da15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.524885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6f3a4-c48d-4364-93f3-d2c61ca4da15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-7fb6f3a4-c48d-4364-93f3-d2c61ca4da15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:18.530370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:18.532030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:18.533222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 101/188] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:18.534949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6f3a4-c48d-4364-93f3-d2c61ca4da15 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.589894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:18.595086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0848243-339f-4009-b7d0-a7ed84949641 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.630613 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8e79d-4f71-4d56-a3bd-ce9ced461efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.631236 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8e79d-4f71-4d56-a3bd-ce9ced461efa None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.631509 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8e79d-4f71-4d56-a3bd-ce9ced461efa None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.631773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8e79d-4f71-4d56-a3bd-ce9ced461efa None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.632917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8e79d-4f71-4d56-a3bd-ce9ced461efa None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:18.634435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8e79d-4f71-4d56-a3bd-ce9ced461efa None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:18.636165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 88/189] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:18.636934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8e79d-4f71-4d56-a3bd-ce9ced461efa None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.649819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6f3a4-c48d-4364-93f3-d2c61ca4da15 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.682915 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fe192-22b7-4677-a729-40a763b14430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.683539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fe192-22b7-4677-a729-40a763b14430 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.683857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fe192-22b7-4677-a729-40a763b14430 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.684127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fe192-22b7-4677-a729-40a763b14430 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.685417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27fe192-22b7-4677-a729-40a763b14430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:18.686870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27fe192-22b7-4677-a729-40a763b14430 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:18.688448 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 102/190] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:18.689236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fe192-22b7-4677-a729-40a763b14430 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.693581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:18.749078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8e79d-4f71-4d56-a3bd-ce9ced461efa None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.787018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eba3bf-c527-4f33-bcbe-7a5a10a371da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.787538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eba3bf-c527-4f33-bcbe-7a5a10a371da None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.787816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eba3bf-c527-4f33-bcbe-7a5a10a371da None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.788088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eba3bf-c527-4f33-bcbe-7a5a10a371da None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.788901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eba3bf-c527-4f33-bcbe-7a5a10a371da None admin] RBAC: Adding query filter params (name=reader) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:18.789205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eba3bf-c527-4f33-bcbe-7a5a10a371da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:18.790756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eba3bf-c527-4f33-bcbe-7a5a10a371da None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:18.800241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 89/191] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:18.805580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fe192-22b7-4677-a729-40a763b14430 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.805704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eba3bf-c527-4f33-bcbe-7a5a10a371da None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.819758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:18.846251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb8f5c-2232-4d50-bb92-5d58a8a239b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.847519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb8f5c-2232-4d50-bb92-5d58a8a239b2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.848030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb8f5c-2232-4d50-bb92-5d58a8a239b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.848499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb8f5c-2232-4d50-bb92-5d58a8a239b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.849488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb8f5c-2232-4d50-bb92-5d58a8a239b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:18.849791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb8f5c-2232-4d50-bb92-5d58a8a239b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:18.851205 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb8f5c-2232-4d50-bb92-5d58a8a239b2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:18.860036 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 103/192] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:18.864741 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb8f5c-2232-4d50-bb92-5d58a8a239b2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.920384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eba3bf-c527-4f33-bcbe-7a5a10a371da None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.938337 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:18.965411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9fd52-46c0-4d3a-83ca-ed0795d6cf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.966193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9fd52-46c0-4d3a-83ca-ed0795d6cf9f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.966878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9fd52-46c0-4d3a-83ca-ed0795d6cf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.966878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9fd52-46c0-4d3a-83ca-ed0795d6cf9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.968123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb8f5c-2232-4d50-bb92-5d58a8a239b2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.981308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9fd52-46c0-4d3a-83ca-ed0795d6cf9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-8eb9fd52-46c0-4d3a-83ca-ed0795d6cf9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:18.986701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:18.987555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:18.990541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 90/193] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:18.992051 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9fd52-46c0-4d3a-83ca-ed0795d6cf9f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.006517 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc104523-8779-4c39-9cf4-ce318d91163a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.007131 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc104523-8779-4c39-9cf4-ce318d91163a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.007410 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc104523-8779-4c39-9cf4-ce318d91163a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.007947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc104523-8779-4c39-9cf4-ce318d91163a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.008396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc104523-8779-4c39-9cf4-ce318d91163a None admin] RBAC: Adding query filter params (role.id=16317a22e9c94a8886f02efb75ab456e, scope.project.id=79572ca8d9504396b675929feb75e9f2, user.id=8563b95970a84781849db7018a225599) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:19.008769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc104523-8779-4c39-9cf4-ce318d91163a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.011668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc104523-8779-4c39-9cf4-ce318d91163a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.024112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 104/194] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/role_assignments?user.id=8563b95970a84781849db7018a225599&scope.project.id=79572ca8d9504396b675929feb75e9f2&role.id=16317a22e9c94a8886f02efb75ab456e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:19.025169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc104523-8779-4c39-9cf4-ce318d91163a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.083470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:19.141475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9fd52-46c0-4d3a-83ca-ed0795d6cf9f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.159682 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc104523-8779-4c39-9cf4-ce318d91163a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.183249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c060e-e9dd-47bf-a90f-a144d380eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.184959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c060e-e9dd-47bf-a90f-a144d380eea4 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.185409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c060e-e9dd-47bf-a90f-a144d380eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.185690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c060e-e9dd-47bf-a90f-a144d380eea4 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.186358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c060e-e9dd-47bf-a90f-a144d380eea4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.187845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c060e-e9dd-47bf-a90f-a144d380eea4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.202786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c50e1-f0a3-4a09-8f11-fba89fc55069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.203419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c50e1-f0a3-4a09-8f11-fba89fc55069 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.203692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c50e1-f0a3-4a09-8f11-fba89fc55069 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.203958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c50e1-f0a3-4a09-8f11-fba89fc55069 None admin] PATH_INFO: `/v3/users/49d9450bae4b4227a60bc3addec76c0e` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.205760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c50e1-f0a3-4a09-8f11-fba89fc55069 None admin] RBAC: Authorizing `identity:get_user(user_id=49d9450bae4b4227a60bc3addec76c0e)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.207531 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c50e1-f0a3-4a09-8f11-fba89fc55069 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.209390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 105/195] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/users/49d9450bae4b4227a60bc3addec76c0e => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:19.210219 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c50e1-f0a3-4a09-8f11-fba89fc55069 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.227435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 91/196] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:18 2022] POST /identity/v3/users => generated 297 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:19.228834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c060e-e9dd-47bf-a90f-a144d380eea4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.276589 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:19.339505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c50e1-f0a3-4a09-8f11-fba89fc55069 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.347872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:19.366595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c060e-e9dd-47bf-a90f-a144d380eea4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.378059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f852bdc-9f0d-46ed-b8d9-ba2aa8cc7bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.378697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f852bdc-9f0d-46ed-b8d9-ba2aa8cc7bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.378975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f852bdc-9f0d-46ed-b8d9-ba2aa8cc7bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.379433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f852bdc-9f0d-46ed-b8d9-ba2aa8cc7bf7 None admin] PATH_INFO: `/v3/users/3c8b48ee00d947abb05c01ae2488d550` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.396193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f852bdc-9f0d-46ed-b8d9-ba2aa8cc7bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8b48ee00d947abb05c01ae2488d550)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.398313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f852bdc-9f0d-46ed-b8d9-ba2aa8cc7bf7 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.404100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 106/197] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/users/3c8b48ee00d947abb05c01ae2488d550 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:19.404100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f852bdc-9f0d-46ed-b8d9-ba2aa8cc7bf7 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.413960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6094c-bb65-40cb-8f7f-570d3e93295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.414662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6094c-bb65-40cb-8f7f-570d3e93295a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.414942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6094c-bb65-40cb-8f7f-570d3e93295a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.415208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6094c-bb65-40cb-8f7f-570d3e93295a None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.415945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6094c-bb65-40cb-8f7f-570d3e93295a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:19.416171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6094c-bb65-40cb-8f7f-570d3e93295a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.417821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6094c-bb65-40cb-8f7f-570d3e93295a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.428000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 92/198] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:19.428953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6094c-bb65-40cb-8f7f-570d3e93295a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.529087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f852bdc-9f0d-46ed-b8d9-ba2aa8cc7bf7 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.549240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:19.558683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6094c-bb65-40cb-8f7f-570d3e93295a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.570369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5bf2b-f08a-4a90-88b7-6a0ea7941cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.571227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5bf2b-f08a-4a90-88b7-6a0ea7941cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.571650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5bf2b-f08a-4a90-88b7-6a0ea7941cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.572072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5bf2b-f08a-4a90-88b7-6a0ea7941cc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.598640 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f78e3d-9140-460e-87be-17b6644cd25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.599304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f78e3d-9140-460e-87be-17b6644cd25a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.599577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f78e3d-9140-460e-87be-17b6644cd25a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.599881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f78e3d-9140-460e-87be-17b6644cd25a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.609189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:19.655118 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5bf2b-f08a-4a90-88b7-6a0ea7941cc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.656604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5bf2b-f08a-4a90-88b7-6a0ea7941cc8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.687687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f78e3d-9140-460e-87be-17b6644cd25a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.689594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f78e3d-9140-460e-87be-17b6644cd25a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.771895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 107/199] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:19.776928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5bf2b-f08a-4a90-88b7-6a0ea7941cc8 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.812792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:19.830149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 93/200] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:19.831454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f78e3d-9140-460e-87be-17b6644cd25a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.854985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:19.900471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5bf2b-f08a-4a90-88b7-6a0ea7941cc8 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.944233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c4fae-7c5b-45b6-a5d1-44d8a6d59ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.945317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c4fae-7c5b-45b6-a5d1-44d8a6d59ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.945697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c4fae-7c5b-45b6-a5d1-44d8a6d59ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.945978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c4fae-7c5b-45b6-a5d1-44d8a6d59ee8 None admin] PATH_INFO: `/v3/users/3c8b48ee00d947abb05c01ae2488d550` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.947099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c4fae-7c5b-45b6-a5d1-44d8a6d59ee8 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8b48ee00d947abb05c01ae2488d550)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.947943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f78e3d-9140-460e-87be-17b6644cd25a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.948577 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c4fae-7c5b-45b6-a5d1-44d8a6d59ee8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.950498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 108/201] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/users/3c8b48ee00d947abb05c01ae2488d550 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:19.951469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c4fae-7c5b-45b6-a5d1-44d8a6d59ee8 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.992712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdd2f3-183d-444f-8374-79356a7df6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.993651 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdd2f3-183d-444f-8374-79356a7df6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.994185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdd2f3-183d-444f-8374-79356a7df6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.994652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdd2f3-183d-444f-8374-79356a7df6b0 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.995989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdd2f3-183d-444f-8374-79356a7df6b0 None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.997929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdd2f3-183d-444f-8374-79356a7df6b0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.999947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 94/202] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:20.001087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdd2f3-183d-444f-8374-79356a7df6b0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:20.066820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c4fae-7c5b-45b6-a5d1-44d8a6d59ee8 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:20.077776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:20.103283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:20.109594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43aeb40-be8e-4c90-b2ff-637bffcfe4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:20.110576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43aeb40-be8e-4c90-b2ff-637bffcfe4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.111133 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43aeb40-be8e-4c90-b2ff-637bffcfe4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.111602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43aeb40-be8e-4c90-b2ff-637bffcfe4d6 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.113012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43aeb40-be8e-4c90-b2ff-637bffcfe4d6 None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:20.114754 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43aeb40-be8e-4c90-b2ff-637bffcfe4d6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:20.117281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 109/203] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:20.118325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43aeb40-be8e-4c90-b2ff-637bffcfe4d6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:20.126285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdd2f3-183d-444f-8374-79356a7df6b0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:20.172587 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa0a5f-db5e-4054-b35c-2c15c873eeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:20.173266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa0a5f-db5e-4054-b35c-2c15c873eeb7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.173489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa0a5f-db5e-4054-b35c-2c15c873eeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.173729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa0a5f-db5e-4054-b35c-2c15c873eeb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.238733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43aeb40-be8e-4c90-b2ff-637bffcfe4d6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:20.255724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa0a5f-db5e-4054-b35c-2c15c873eeb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:20.257587 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa0a5f-db5e-4054-b35c-2c15c873eeb7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:20.273357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886109c-51d3-46d5-bd66-a2d167bacc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:20.273681 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886109c-51d3-46d5-bd66-a2d167bacc76 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.273957 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886109c-51d3-46d5-bd66-a2d167bacc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.274226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886109c-51d3-46d5-bd66-a2d167bacc76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.274945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886109c-51d3-46d5-bd66-a2d167bacc76 None admin] RBAC: Adding query filter params (role.id=26063cb085b6415989a19c8e7dc2c3c5, scope.project.id=79572ca8d9504396b675929feb75e9f2, user.id=49d9450bae4b4227a60bc3addec76c0e) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:20.275295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886109c-51d3-46d5-bd66-a2d167bacc76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:20.276688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886109c-51d3-46d5-bd66-a2d167bacc76 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:20.282706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 110/204] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:20 2022] GET /identity/v3/role_assignments?user.id=49d9450bae4b4227a60bc3addec76c0e&scope.project.id=79572ca8d9504396b675929feb75e9f2&role.id=26063cb085b6415989a19c8e7dc2c3c5 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:20.283507 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886109c-51d3-46d5-bd66-a2d167bacc76 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:20.328841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:20.350810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:20.392479 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886109c-51d3-46d5-bd66-a2d167bacc76 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:20.393541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 95/205] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:20.399256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa0a5f-db5e-4054-b35c-2c15c873eeb7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:20.429099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc375b-f0eb-467c-aa54-c144790fc9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:20.429707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc375b-f0eb-467c-aa54-c144790fc9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.429982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc375b-f0eb-467c-aa54-c144790fc9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.430256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc375b-f0eb-467c-aa54-c144790fc9f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.430979 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc375b-f0eb-467c-aa54-c144790fc9f8 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=9549a5efb25e493d9a9b245fcc802166, user.id=3c8b48ee00d947abb05c01ae2488d550) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:20.431282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc375b-f0eb-467c-aa54-c144790fc9f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:20.432763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc375b-f0eb-467c-aa54-c144790fc9f8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:20.439670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 111/206] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:20 2022] GET /identity/v3/role_assignments?user.id=3c8b48ee00d947abb05c01ae2488d550&scope.project.id=9549a5efb25e493d9a9b245fcc802166&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:20.573003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa0a5f-db5e-4054-b35c-2c15c873eeb7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:20.613087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c2711-f625-48c2-858a-f39f7092acb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:20.613826 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c2711-f625-48c2-858a-f39f7092acb7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.614098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c2711-f625-48c2-858a-f39f7092acb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.614364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c2711-f625-48c2-858a-f39f7092acb7 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.615489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c2711-f625-48c2-858a-f39f7092acb7 None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:20.617076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c2711-f625-48c2-858a-f39f7092acb7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:20.619018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 96/207] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:20 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:20.624907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc375b-f0eb-467c-aa54-c144790fc9f8 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:20.730322 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc375b-f0eb-467c-aa54-c144790fc9f8 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:20.757672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:20.766125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75827eac-5b48-4157-a33b-1048cc389b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:20.766820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75827eac-5b48-4157-a33b-1048cc389b11 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.767156 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75827eac-5b48-4157-a33b-1048cc389b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.767482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75827eac-5b48-4157-a33b-1048cc389b11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.768266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75827eac-5b48-4157-a33b-1048cc389b11 None admin] RBAC: Adding query filter params (role.id=16317a22e9c94a8886f02efb75ab456e, scope.project.id=9549a5efb25e493d9a9b245fcc802166, user.id=3c8b48ee00d947abb05c01ae2488d550) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:20.768629 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75827eac-5b48-4157-a33b-1048cc389b11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:20.770150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75827eac-5b48-4157-a33b-1048cc389b11 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:20.775939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 112/208] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:20 2022] GET /identity/v3/role_assignments?user.id=3c8b48ee00d947abb05c01ae2488d550&scope.project.id=9549a5efb25e493d9a9b245fcc802166&role.id=16317a22e9c94a8886f02efb75ab456e => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:21.443593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23cb8d-9827-4ab4-b986-8d3314969ee1 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.444313 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23cb8d-9827-4ab4-b986-8d3314969ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.444313 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23cb8d-9827-4ab4-b986-8d3314969ee1 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.445740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 97/209] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:21.451517 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fb7d-a32a-4162-abf2-bd6a8c92de9f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.452053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fb7d-a32a-4162-abf2-bd6a8c92de9f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.452053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fb7d-a32a-4162-abf2-bd6a8c92de9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.471186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-dd92fb7d-a32a-4162-abf2-bd6a8c92de9f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:21.472391 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dd92fb7d-a32a-4162-abf2-bd6a8c92de9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:21.488692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906ebe6-8f91-4304-8deb-f9c7a5a8ba1b None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.488692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906ebe6-8f91-4304-8deb-f9c7a5a8ba1b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.488692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906ebe6-8f91-4304-8deb-f9c7a5a8ba1b None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.490230 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 98/210] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:21.495743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995fe7c-1d49-4729-8fe4-f7796ae4fc97 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.496255 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995fe7c-1d49-4729-8fe4-f7796ae4fc97 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.496943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995fe7c-1d49-4729-8fe4-f7796ae4fc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.516591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2995fe7c-1d49-4729-8fe4-f7796ae4fc97 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:21.518223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2995fe7c-1d49-4729-8fe4-f7796ae4fc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:21.542417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 113/211] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:21.553199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:21.596928 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 99/212] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:21.812453 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92fb7d-a32a-4162-abf2-bd6a8c92de9f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:21.866319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995fe7c-1d49-4729-8fe4-f7796ae4fc97 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:21.918084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92fb7d-a32a-4162-abf2-bd6a8c92de9f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:21.941480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:21.953654 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc7b0f-9e72-4931-b6c4-308a264ef526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:21.954297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc7b0f-9e72-4931-b6c4-308a264ef526 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.954571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc7b0f-9e72-4931-b6c4-308a264ef526 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.954839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc7b0f-9e72-4931-b6c4-308a264ef526 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.955940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc7b0f-9e72-4931-b6c4-308a264ef526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:21.957441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc7b0f-9e72-4931-b6c4-308a264ef526 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:21.959314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 114/213] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:21.965101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc7b0f-9e72-4931-b6c4-308a264ef526 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:21.984693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995fe7c-1d49-4729-8fe4-f7796ae4fc97 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:22.018389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d96f7-cda1-4151-aec0-9fd7ac50f4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:22.019270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d96f7-cda1-4151-aec0-9fd7ac50f4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.019640 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d96f7-cda1-4151-aec0-9fd7ac50f4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.020019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d96f7-cda1-4151-aec0-9fd7ac50f4f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.034709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d96f7-cda1-4151-aec0-9fd7ac50f4f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-023d96f7-cda1-4151-aec0-9fd7ac50f4f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:22.042503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:22.043768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:22.044685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:22.044685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:22.044685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:22.044685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:22.044685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:22.044685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:22.044685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:22.044685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:22.044685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:22.044685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:22.044685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:22.044685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:22.045286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 100/214] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:22.052552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d96f7-cda1-4151-aec0-9fd7ac50f4f0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:22.073296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc7b0f-9e72-4931-b6c4-308a264ef526 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:22.079942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:22.109564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d207ee-5eb9-4a69-8c25-abcc9939f67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:22.110321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d207ee-5eb9-4a69-8c25-abcc9939f67b None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.110692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d207ee-5eb9-4a69-8c25-abcc9939f67b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.111072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d207ee-5eb9-4a69-8c25-abcc9939f67b None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.111869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d207ee-5eb9-4a69-8c25-abcc9939f67b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:22.113447 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d207ee-5eb9-4a69-8c25-abcc9939f67b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:22.148751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 115/215] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:21 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:22.165820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d96f7-cda1-4151-aec0-9fd7ac50f4f0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:22.208992 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3da9a5-9c86-4d58-96e2-bde3fe98a2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:22.209662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3da9a5-9c86-4d58-96e2-bde3fe98a2fc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.209943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3da9a5-9c86-4d58-96e2-bde3fe98a2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.210212 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3da9a5-9c86-4d58-96e2-bde3fe98a2fc None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.210897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3da9a5-9c86-4d58-96e2-bde3fe98a2fc None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:22.211198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3da9a5-9c86-4d58-96e2-bde3fe98a2fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:22.212714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3da9a5-9c86-4d58-96e2-bde3fe98a2fc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:22.227625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 101/216] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:22.229780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d207ee-5eb9-4a69-8c25-abcc9939f67b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:22.282885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:22.346853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d207ee-5eb9-4a69-8c25-abcc9939f67b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:22.382138 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93bb4a-683f-4b88-9c7f-452bfc42a18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:22.382827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93bb4a-683f-4b88-9c7f-452bfc42a18e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.383091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93bb4a-683f-4b88-9c7f-452bfc42a18e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.383354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93bb4a-683f-4b88-9c7f-452bfc42a18e None admin] PATH_INFO: `/v3/users/dcd2ab21c0804e658b41f9945b48667f` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.398705 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93bb4a-683f-4b88-9c7f-452bfc42a18e None admin] RBAC: Authorizing `identity:get_user(user_id=dcd2ab21c0804e658b41f9945b48667f)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:22.400257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93bb4a-683f-4b88-9c7f-452bfc42a18e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:22.404786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 116/217] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:22 2022] GET /identity/v3/users/dcd2ab21c0804e658b41f9945b48667f => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:22.408306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3da9a5-9c86-4d58-96e2-bde3fe98a2fc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:22.417141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f056b0-d766-4c59-b94b-1b85ca7e0c93 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.417413 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f056b0-d766-4c59-b94b-1b85ca7e0c93 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.417684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f056b0-d766-4c59-b94b-1b85ca7e0c93 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.419201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 117/218] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:22.426010 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424f0d3-7f2c-436f-a287-f81f6b020c0c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.426289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424f0d3-7f2c-436f-a287-f81f6b020c0c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.426563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424f0d3-7f2c-436f-a287-f81f6b020c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.450318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a424f0d3-7f2c-436f-a287-f81f6b020c0c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:22.451519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a424f0d3-7f2c-436f-a287-f81f6b020c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:22.528368 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 118/219] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:22.541747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:22.559116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3da9a5-9c86-4d58-96e2-bde3fe98a2fc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:22.595675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6faedc-b86e-4b5a-959e-152fb479be81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:22.596279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6faedc-b86e-4b5a-959e-152fb479be81 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.596552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6faedc-b86e-4b5a-959e-152fb479be81 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.596850 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6faedc-b86e-4b5a-959e-152fb479be81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.690718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6faedc-b86e-4b5a-959e-152fb479be81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:22.692123 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6faedc-b86e-4b5a-959e-152fb479be81 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:22.785298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6eb1b-d911-4dfe-b40d-72c8fa0d671f None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.785951 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6eb1b-d911-4dfe-b40d-72c8fa0d671f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.786412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6eb1b-d911-4dfe-b40d-72c8fa0d671f None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.788213 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 119/220] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:22.792913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6c0ab-6223-49f9-b701-e8769c336748 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.794155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6c0ab-6223-49f9-b701-e8769c336748 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.794860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6c0ab-6223-49f9-b701-e8769c336748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.816599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-26f6c0ab-6223-49f9-b701-e8769c336748 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:22.818132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-26f6c0ab-6223-49f9-b701-e8769c336748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:22.824851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:22.859943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:22.860129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 102/221] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:22.860976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6faedc-b86e-4b5a-959e-152fb479be81 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:22.918227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 120/222] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:22.919013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6c0ab-6223-49f9-b701-e8769c336748 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:22.997897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6faedc-b86e-4b5a-959e-152fb479be81 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:23.020448 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6c0ab-6223-49f9-b701-e8769c336748 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:23.045392 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e010ad1-06a4-41b5-9cc0-5103dfc94aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:23.046531 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e010ad1-06a4-41b5-9cc0-5103dfc94aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:23.046999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e010ad1-06a4-41b5-9cc0-5103dfc94aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:23.047454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e010ad1-06a4-41b5-9cc0-5103dfc94aa6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:23.060693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580f87-7521-4968-88cb-e2b7a0cbc377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:23.060693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580f87-7521-4968-88cb-e2b7a0cbc377 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:23.060693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580f87-7521-4968-88cb-e2b7a0cbc377 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:23.060693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580f87-7521-4968-88cb-e2b7a0cbc377 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:23.062612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe580f87-7521-4968-88cb-e2b7a0cbc377 None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:23.064095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe580f87-7521-4968-88cb-e2b7a0cbc377 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:23.068202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 121/223] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:22 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:23.068970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e010ad1-06a4-41b5-9cc0-5103dfc94aa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:23.073723 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580f87-7521-4968-88cb-e2b7a0cbc377 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-6e010ad1-06a4-41b5-9cc0-5103dfc94aa6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:23.078264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:23.079139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:23.079938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:23.079938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:23.079938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:23.079938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:23.079938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:23.079938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:23.079938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:23.079938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:23.079938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:23.079938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:23.079938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:23.079938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:23.084808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 103/224] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:23.090806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e010ad1-06a4-41b5-9cc0-5103dfc94aa6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:23.137216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:23.189084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:23.259862 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580f87-7521-4968-88cb-e2b7a0cbc377 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:23.268162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e010ad1-06a4-41b5-9cc0-5103dfc94aa6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:23.301308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b817cee-3a34-47f0-b854-303c7373aed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:23.301916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b817cee-3a34-47f0-b854-303c7373aed5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:23.302194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b817cee-3a34-47f0-b854-303c7373aed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:23.302462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b817cee-3a34-47f0-b854-303c7373aed5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:23.303191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b817cee-3a34-47f0-b854-303c7373aed5 None admin] RBAC: Adding query filter params (role.id=3647def0715f4fd4991f82b06815ee38, scope.project.id=9549a5efb25e493d9a9b245fcc802166, user.id=dcd2ab21c0804e658b41f9945b48667f) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:23.303494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b817cee-3a34-47f0-b854-303c7373aed5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:23.304983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b817cee-3a34-47f0-b854-303c7373aed5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:23.306030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8fb6a-b5e4-4650-8149-6ccd31ef3852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:23.306699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8fb6a-b5e4-4650-8149-6ccd31ef3852 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:23.306984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8fb6a-b5e4-4650-8149-6ccd31ef3852 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:23.307260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8fb6a-b5e4-4650-8149-6ccd31ef3852 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:23.307934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8fb6a-b5e4-4650-8149-6ccd31ef3852 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:23.308240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8fb6a-b5e4-4650-8149-6ccd31ef3852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:23.309781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8fb6a-b5e4-4650-8149-6ccd31ef3852 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:23.311634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 122/225] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:23 2022] GET /identity/v3/role_assignments?user.id=dcd2ab21c0804e658b41f9945b48667f&scope.project.id=9549a5efb25e493d9a9b245fcc802166&role.id=3647def0715f4fd4991f82b06815ee38 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:23.318110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b817cee-3a34-47f0-b854-303c7373aed5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:23.319839 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 104/226] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:23.325384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8fb6a-b5e4-4650-8149-6ccd31ef3852 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:23.509231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:23.512121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8fb6a-b5e4-4650-8149-6ccd31ef3852 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:23.517322 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b817cee-3a34-47f0-b854-303c7373aed5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:23.521149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:23.565839 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1301d-6bcd-453f-b122-5741d89054b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:23.567972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1301d-6bcd-453f-b122-5741d89054b3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:23.568983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1301d-6bcd-453f-b122-5741d89054b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:23.569604 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1301d-6bcd-453f-b122-5741d89054b3 None admin] PATH_INFO: `/v3/users/3c8b48ee00d947abb05c01ae2488d550` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:23.571682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1301d-6bcd-453f-b122-5741d89054b3 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8b48ee00d947abb05c01ae2488d550)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:23.573804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1301d-6bcd-453f-b122-5741d89054b3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:23.576191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 105/227] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:23 2022] GET /identity/v3/users/3c8b48ee00d947abb05c01ae2488d550 => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:23.582553 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cefb4bc-c2b7-49de-8c03-a513e8af9891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:23.583758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefb4bc-c2b7-49de-8c03-a513e8af9891 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:23.584207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefb4bc-c2b7-49de-8c03-a513e8af9891 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:23.584689 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefb4bc-c2b7-49de-8c03-a513e8af9891 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:23.586909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1301d-6bcd-453f-b122-5741d89054b3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:23.607380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cefb4bc-c2b7-49de-8c03-a513e8af9891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-4cefb4bc-c2b7-49de-8c03-a513e8af9891 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:23.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:23.615566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:23.616349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:23.616349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:23.616349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:23.616349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:23.616349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:23.616349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:23.616349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:23.616349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:23.616349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:23.616349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:23.616349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:23.616349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:23.626092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 123/228] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:23.629425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cefb4bc-c2b7-49de-8c03-a513e8af9891 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:23.787239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1301d-6bcd-453f-b122-5741d89054b3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:23.829761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cefb4bc-c2b7-49de-8c03-a513e8af9891 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:23.841816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a758da5-101d-4cbb-a37c-5b21f9bfa613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:23.842464 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a758da5-101d-4cbb-a37c-5b21f9bfa613 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:23.842742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a758da5-101d-4cbb-a37c-5b21f9bfa613 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:23.843013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a758da5-101d-4cbb-a37c-5b21f9bfa613 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:23.844837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a758da5-101d-4cbb-a37c-5b21f9bfa613 None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:23.846367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a758da5-101d-4cbb-a37c-5b21f9bfa613 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:23.850617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 106/229] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:23 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:23.855620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a758da5-101d-4cbb-a37c-5b21f9bfa613 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:23.897402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfc57d-b440-4686-804f-efabb90c9cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:23.898020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfc57d-b440-4686-804f-efabb90c9cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:23.898300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfc57d-b440-4686-804f-efabb90c9cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:23.898574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfc57d-b440-4686-804f-efabb90c9cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:23.899246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfc57d-b440-4686-804f-efabb90c9cd8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:23.899550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfc57d-b440-4686-804f-efabb90c9cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:23.900996 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfc57d-b440-4686-804f-efabb90c9cd8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:23.910419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 124/230] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:23.916819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfc57d-b440-4686-804f-efabb90c9cd8 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:23.942319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:23.954958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:24.035243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a758da5-101d-4cbb-a37c-5b21f9bfa613 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:24.070189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9334d5-dd0b-4ce6-82f3-ed789d5d26bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:24.070769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9334d5-dd0b-4ce6-82f3-ed789d5d26bc None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.071044 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9334d5-dd0b-4ce6-82f3-ed789d5d26bc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.071317 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9334d5-dd0b-4ce6-82f3-ed789d5d26bc None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166/users/3c8b48ee00d947abb05c01ae2488d550/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.073323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9334d5-dd0b-4ce6-82f3-ed789d5d26bc None admin] RBAC: Authorizing `identity:create_grant(project_id=9549a5efb25e493d9a9b245fcc802166, user_id=3c8b48ee00d947abb05c01ae2488d550, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:24.074880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9334d5-dd0b-4ce6-82f3-ed789d5d26bc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:24.100393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 107/231] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:23 2022] PUT /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166/users/3c8b48ee00d947abb05c01ae2488d550/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:19:24.112503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfc57d-b440-4686-804f-efabb90c9cd8 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:24.166615 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d3ce5-6e7a-4aba-a2aa-d488b340f012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:24.167289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d3ce5-6e7a-4aba-a2aa-d488b340f012 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.167567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d3ce5-6e7a-4aba-a2aa-d488b340f012 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.167844 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d3ce5-6e7a-4aba-a2aa-d488b340f012 None admin] PATH_INFO: `/v3/users/3c8b48ee00d947abb05c01ae2488d550` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.169555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d3ce5-6e7a-4aba-a2aa-d488b340f012 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8b48ee00d947abb05c01ae2488d550)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:24.171079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d3ce5-6e7a-4aba-a2aa-d488b340f012 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:24.173224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 125/232] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:23 2022] GET /identity/v3/users/3c8b48ee00d947abb05c01ae2488d550 => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:24.181904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9334d5-dd0b-4ce6-82f3-ed789d5d26bc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:24.321157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9334d5-dd0b-4ce6-82f3-ed789d5d26bc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:24.322692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:24.359504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe99c99-9865-4a88-9aa3-229a7b6d2fd6 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.360162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe99c99-9865-4a88-9aa3-229a7b6d2fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.360631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe99c99-9865-4a88-9aa3-229a7b6d2fd6 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.362308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 126/233] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:24.368494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60545e44-ea3b-4cb0-99c1-5e1cf64461c3 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.369091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60545e44-ea3b-4cb0-99c1-5e1cf64461c3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.369576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60545e44-ea3b-4cb0-99c1-5e1cf64461c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.373567 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6078fd4-6f69-4298-86e9-aafcbee14173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:24.374261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6078fd4-6f69-4298-86e9-aafcbee14173 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.374538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6078fd4-6f69-4298-86e9-aafcbee14173 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.374809 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6078fd4-6f69-4298-86e9-aafcbee14173 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.376089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6078fd4-6f69-4298-86e9-aafcbee14173 None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:24.377735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6078fd4-6f69-4298-86e9-aafcbee14173 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:24.379819 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 108/234] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:24 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:24.385424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6078fd4-6f69-4298-86e9-aafcbee14173 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:24.391741 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-60545e44-ea3b-4cb0-99c1-5e1cf64461c3 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:24.393301 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-60545e44-ea3b-4cb0-99c1-5e1cf64461c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:24.482306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 127/235] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:24.531656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6078fd4-6f69-4298-86e9-aafcbee14173 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:24.576233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dfda0-3d2d-4a60-ad4e-98852fdf0240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:24.577710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dfda0-3d2d-4a60-ad4e-98852fdf0240 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.578201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dfda0-3d2d-4a60-ad4e-98852fdf0240 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.578666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dfda0-3d2d-4a60-ad4e-98852fdf0240 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166/users/3c8b48ee00d947abb05c01ae2488d550/roles/16317a22e9c94a8886f02efb75ab456e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.582393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dfda0-3d2d-4a60-ad4e-98852fdf0240 None admin] RBAC: Authorizing `identity:create_grant(project_id=9549a5efb25e493d9a9b245fcc802166, user_id=3c8b48ee00d947abb05c01ae2488d550, role_id=16317a22e9c94a8886f02efb75ab456e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:24.584092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dfda0-3d2d-4a60-ad4e-98852fdf0240 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:24.613421 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 109/236] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:24 2022] PUT /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166/users/3c8b48ee00d947abb05c01ae2488d550/roles/16317a22e9c94a8886f02efb75ab456e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:19:24.774834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956be310-b778-4400-b43b-07e91e628d23 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.775345 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956be310-b778-4400-b43b-07e91e628d23 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.775345 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956be310-b778-4400-b43b-07e91e628d23 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.776733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dfda0-3d2d-4a60-ad4e-98852fdf0240 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:24.776839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 128/237] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:24.782099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bae25a-1ce8-42a1-8084-0030ce4bcaeb None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.782351 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bae25a-1ce8-42a1-8084-0030ce4bcaeb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.782579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bae25a-1ce8-42a1-8084-0030ce4bcaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.813612 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:24.813801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b5bae25a-1ce8-42a1-8084-0030ce4bcaeb None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:24.814926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b5bae25a-1ce8-42a1-8084-0030ce4bcaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:24.853905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:24.905885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 129/238] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:24.924515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dfda0-3d2d-4a60-ad4e-98852fdf0240 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:24.960021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5aabc-8964-47f3-8746-3fb10ebeeb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:24.961091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5aabc-8964-47f3-8746-3fb10ebeeb8d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.961611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5aabc-8964-47f3-8746-3fb10ebeeb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.962069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5aabc-8964-47f3-8746-3fb10ebeeb8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.973214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5aabc-8964-47f3-8746-3fb10ebeeb8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-efe5aabc-8964-47f3-8746-3fb10ebeeb8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:24.979253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:24.980109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:24.981038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:24.981038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:24.981038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:24.981038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:24.981038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:24.981038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:24.981038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:24.981038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:24.981038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:24.981038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:24.981038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:24.981038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:24.988152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 110/239] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:24.989767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bae25a-1ce8-42a1-8084-0030ce4bcaeb None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.097039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bae25a-1ce8-42a1-8084-0030ce4bcaeb None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:25.134516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ce652-9810-41d3-9d51-1d5ebecb073b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:25.135524 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ce652-9810-41d3-9d51-1d5ebecb073b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.136009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ce652-9810-41d3-9d51-1d5ebecb073b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.136506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ce652-9810-41d3-9d51-1d5ebecb073b None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.137632 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ce652-9810-41d3-9d51-1d5ebecb073b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:25.138130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ce652-9810-41d3-9d51-1d5ebecb073b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:25.139734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ce652-9810-41d3-9d51-1d5ebecb073b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:25.149158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 130/240] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:25.155233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5aabc-8964-47f3-8746-3fb10ebeeb8d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.251397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:25.261860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5aabc-8964-47f3-8746-3fb10ebeeb8d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:25.262285 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ce652-9810-41d3-9d51-1d5ebecb073b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.275396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:25.308003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36069de-2647-4b9a-9828-85f6d1552e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:25.309929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36069de-2647-4b9a-9828-85f6d1552e7f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.310406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36069de-2647-4b9a-9828-85f6d1552e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.310892 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36069de-2647-4b9a-9828-85f6d1552e7f None admin] PATH_INFO: `/v3/users/7312047a0e204706b67cec86de5fa181` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.329000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36069de-2647-4b9a-9828-85f6d1552e7f None admin] RBAC: Authorizing `identity:get_user(user_id=7312047a0e204706b67cec86de5fa181)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:25.331792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36069de-2647-4b9a-9828-85f6d1552e7f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:25.334972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 111/241] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:25 2022] GET /identity/v3/users/7312047a0e204706b67cec86de5fa181 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:25.342059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36069de-2647-4b9a-9828-85f6d1552e7f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.393224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ce652-9810-41d3-9d51-1d5ebecb073b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:25.430777 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca35d9-352a-4c93-b7b8-51c519980d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:25.431532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca35d9-352a-4c93-b7b8-51c519980d17 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.432533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca35d9-352a-4c93-b7b8-51c519980d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.432533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca35d9-352a-4c93-b7b8-51c519980d17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.433467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca35d9-352a-4c93-b7b8-51c519980d17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:25.435713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca35d9-352a-4c93-b7b8-51c519980d17 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:25.437938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 131/242] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:25.443324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca35d9-352a-4c93-b7b8-51c519980d17 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.470975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36069de-2647-4b9a-9828-85f6d1552e7f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:25.513803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a119353-ab14-409d-b1fe-4b69de3f1b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:25.515059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a119353-ab14-409d-b1fe-4b69de3f1b6d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.515644 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a119353-ab14-409d-b1fe-4b69de3f1b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.516191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a119353-ab14-409d-b1fe-4b69de3f1b6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.531908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:25.539040 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:25.564966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca35d9-352a-4c93-b7b8-51c519980d17 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:25.603278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a119353-ab14-409d-b1fe-4b69de3f1b6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:25.606248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a119353-ab14-409d-b1fe-4b69de3f1b6d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:25.611686 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00bac9-19a4-4f84-a072-6b4f71dade82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:25.612381 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00bac9-19a4-4f84-a072-6b4f71dade82 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.612755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00bac9-19a4-4f84-a072-6b4f71dade82 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.613095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00bac9-19a4-4f84-a072-6b4f71dade82 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.613837 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00bac9-19a4-4f84-a072-6b4f71dade82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:25.615378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00bac9-19a4-4f84-a072-6b4f71dade82 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:25.656251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 132/243] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:25 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:25.740330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bacbe8-042d-4cfe-956f-d218eb509aa6 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.741092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bacbe8-042d-4cfe-956f-d218eb509aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.741380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bacbe8-042d-4cfe-956f-d218eb509aa6 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.743217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 133/244] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:25.753271 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa043f-ce2e-4675-9773-a833a42e7a4a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.753579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa043f-ce2e-4675-9773-a833a42e7a4a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.753829 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa043f-ce2e-4675-9773-a833a42e7a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.789787 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 112/245] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:25.791207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2faa043f-ce2e-4675-9773-a833a42e7a4a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:25.793248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2faa043f-ce2e-4675-9773-a833a42e7a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:25.795336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a119353-ab14-409d-b1fe-4b69de3f1b6d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.884215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:25.916463 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 134/246] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:25.955608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a119353-ab14-409d-b1fe-4b69de3f1b6d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.001016 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab33c9c-4e1e-4900-87c4-b4c05922f794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.001667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab33c9c-4e1e-4900-87c4-b4c05922f794 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.001943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab33c9c-4e1e-4900-87c4-b4c05922f794 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.002212 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab33c9c-4e1e-4900-87c4-b4c05922f794 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.004099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab33c9c-4e1e-4900-87c4-b4c05922f794 None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:26.005608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab33c9c-4e1e-4900-87c4-b4c05922f794 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:26.007293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 113/247] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:25 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:26.013976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa043f-ce2e-4675-9773-a833a42e7a4a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:26.081263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:26.127534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa043f-ce2e-4675-9773-a833a42e7a4a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.166966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed07f-9c96-4707-a242-2e3089bd6981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.167504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed07f-9c96-4707-a242-2e3089bd6981 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.167773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed07f-9c96-4707-a242-2e3089bd6981 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.168043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed07f-9c96-4707-a242-2e3089bd6981 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.169086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ed07f-9c96-4707-a242-2e3089bd6981 None admin] RBAC: Adding query filter params (role.id=26063cb085b6415989a19c8e7dc2c3c5, scope.project.id=9549a5efb25e493d9a9b245fcc802166, user.id=7312047a0e204706b67cec86de5fa181) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:26.169194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ed07f-9c96-4707-a242-2e3089bd6981 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:26.170606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ed07f-9c96-4707-a242-2e3089bd6981 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:26.176301 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 135/248] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:26 2022] GET /identity/v3/role_assignments?user.id=7312047a0e204706b67cec86de5fa181&scope.project.id=9549a5efb25e493d9a9b245fcc802166&role.id=26063cb085b6415989a19c8e7dc2c3c5 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:26.220997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab33c9c-4e1e-4900-87c4-b4c05922f794 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:26.369549 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab33c9c-4e1e-4900-87c4-b4c05922f794 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.391771 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:26.411683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41baaa-af16-4dd8-a579-b18fef12364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.412323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41baaa-af16-4dd8-a579-b18fef12364f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.412597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41baaa-af16-4dd8-a579-b18fef12364f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.412895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41baaa-af16-4dd8-a579-b18fef12364f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.423894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41baaa-af16-4dd8-a579-b18fef12364f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-0d41baaa-af16-4dd8-a579-b18fef12364f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:26.428786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:26.429662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:26.430440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 114/249] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:26.435788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed07f-9c96-4707-a242-2e3089bd6981 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:26.551906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed07f-9c96-4707-a242-2e3089bd6981 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.579141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:26.589816 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0345b71-e0d1-4ce4-a5cb-c67829d7e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.590964 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0345b71-e0d1-4ce4-a5cb-c67829d7e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.591075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0345b71-e0d1-4ce4-a5cb-c67829d7e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.591255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0345b71-e0d1-4ce4-a5cb-c67829d7e7a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.591931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0345b71-e0d1-4ce4-a5cb-c67829d7e7a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:26.592230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0345b71-e0d1-4ce4-a5cb-c67829d7e7a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:26.593665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0345b71-e0d1-4ce4-a5cb-c67829d7e7a8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:26.604121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 136/250] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:26.613855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41baaa-af16-4dd8-a579-b18fef12364f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:26.725830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41baaa-af16-4dd8-a579-b18fef12364f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.768312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b635f1d-dc1b-4ae7-a951-f2d4fd3e34c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.769368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b635f1d-dc1b-4ae7-a951-f2d4fd3e34c9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.769846 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b635f1d-dc1b-4ae7-a951-f2d4fd3e34c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.770308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b635f1d-dc1b-4ae7-a951-f2d4fd3e34c9 None admin] PATH_INFO: `/v3/users/dcd2ab21c0804e658b41f9945b48667f` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.771679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b635f1d-dc1b-4ae7-a951-f2d4fd3e34c9 None admin] RBAC: Authorizing `identity:get_user(user_id=dcd2ab21c0804e658b41f9945b48667f)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:26.773409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b635f1d-dc1b-4ae7-a951-f2d4fd3e34c9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:26.775540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 115/251] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:26 2022] GET /identity/v3/users/dcd2ab21c0804e658b41f9945b48667f => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:26.781195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0345b71-e0d1-4ce4-a5cb-c67829d7e7a8 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:26.938010 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0345b71-e0d1-4ce4-a5cb-c67829d7e7a8 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.978372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab10c27-098f-4075-b941-0e606b9734e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.978968 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab10c27-098f-4075-b941-0e606b9734e0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.979248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab10c27-098f-4075-b941-0e606b9734e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.979516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab10c27-098f-4075-b941-0e606b9734e0 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.980772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab10c27-098f-4075-b941-0e606b9734e0 None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:26.982289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab10c27-098f-4075-b941-0e606b9734e0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:26.986137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 137/252] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:26 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:26.990796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b635f1d-dc1b-4ae7-a951-f2d4fd3e34c9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:27.077158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:27.080131 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dee3ca-9631-48de-b2ff-8d93205251cb None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.080424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dee3ca-9631-48de-b2ff-8d93205251cb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.081037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dee3ca-9631-48de-b2ff-8d93205251cb None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.083111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 138/253] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:27.087302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75988d38-5177-4742-ad79-7423018531ce None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.087590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75988d38-5177-4742-ad79-7423018531ce None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.087969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75988d38-5177-4742-ad79-7423018531ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.120798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-75988d38-5177-4742-ad79-7423018531ce None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:27.122069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-75988d38-5177-4742-ad79-7423018531ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:27.139235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b635f1d-dc1b-4ae7-a951-f2d4fd3e34c9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:27.201411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbc31d-0f24-4868-8b0a-01e43014e3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:27.202104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbc31d-0f24-4868-8b0a-01e43014e3a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.202387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbc31d-0f24-4868-8b0a-01e43014e3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.202664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbc31d-0f24-4868-8b0a-01e43014e3a1 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166/users/dcd2ab21c0804e658b41f9945b48667f/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.204704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbc31d-0f24-4868-8b0a-01e43014e3a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9549a5efb25e493d9a9b245fcc802166, user_id=dcd2ab21c0804e658b41f9945b48667f, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:27.206188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbc31d-0f24-4868-8b0a-01e43014e3a1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:27.210072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:27.239288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 139/254] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:27.239628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 116/255] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:26 2022] PUT /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166/users/dcd2ab21c0804e658b41f9945b48667f/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:27.318273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc019d-32d8-4fdb-94dd-2bcff33b0a47 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.318600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc019d-32d8-4fdb-94dd-2bcff33b0a47 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.318825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc019d-32d8-4fdb-94dd-2bcff33b0a47 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.320472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 140/256] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:27.327137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2bd13-6766-40b4-8307-be721e74468f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.327489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2bd13-6766-40b4-8307-be721e74468f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.327682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2bd13-6766-40b4-8307-be721e74468f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.362355 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b9f2bd13-6766-40b4-8307-be721e74468f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:27.363663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b9f2bd13-6766-40b4-8307-be721e74468f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:27.469916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 117/257] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:27.547148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc019d-32d8-4fdb-94dd-2bcff33b0a47 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:27.661705 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc019d-32d8-4fdb-94dd-2bcff33b0a47 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:27.702937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0234978-f630-40e2-b75f-964d711b22be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:27.703576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0234978-f630-40e2-b75f-964d711b22be None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.703899 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0234978-f630-40e2-b75f-964d711b22be None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.704179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0234978-f630-40e2-b75f-964d711b22be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.717161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0234978-f630-40e2-b75f-964d711b22be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-d0234978-f630-40e2-b75f-964d711b22be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:27.721962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:27.722809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:27.723581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 141/258] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:27.733821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2bd13-6766-40b4-8307-be721e74468f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:27.777754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:27.873131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2bd13-6766-40b4-8307-be721e74468f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:27.922468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234af89-7b14-4d7d-889b-9c99f515b4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:27.923464 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234af89-7b14-4d7d-889b-9c99f515b4ed None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.923935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234af89-7b14-4d7d-889b-9c99f515b4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.924387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234af89-7b14-4d7d-889b-9c99f515b4ed None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.925263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234af89-7b14-4d7d-889b-9c99f515b4ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:27.925765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234af89-7b14-4d7d-889b-9c99f515b4ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:27.927348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234af89-7b14-4d7d-889b-9c99f515b4ed None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:27.940766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 118/259] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:27.946019 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0234978-f630-40e2-b75f-964d711b22be None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:27.978820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234af89-7b14-4d7d-889b-9c99f515b4ed None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:28.015923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:28.147839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0234978-f630-40e2-b75f-964d711b22be None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:28.203632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:28.219097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234af89-7b14-4d7d-889b-9c99f515b4ed None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:28.235733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37088da-0a02-4483-8cb8-1b3300aef833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:28.236366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37088da-0a02-4483-8cb8-1b3300aef833 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.236639 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37088da-0a02-4483-8cb8-1b3300aef833 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.245016 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37088da-0a02-4483-8cb8-1b3300aef833 None admin] PATH_INFO: `/v3/users/3c8b48ee00d947abb05c01ae2488d550` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.246275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37088da-0a02-4483-8cb8-1b3300aef833 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8b48ee00d947abb05c01ae2488d550)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:28.247836 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37088da-0a02-4483-8cb8-1b3300aef833 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:28.249925 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 142/260] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:27 2022] GET /identity/v3/users/3c8b48ee00d947abb05c01ae2488d550 => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:28.256064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37088da-0a02-4483-8cb8-1b3300aef833 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:28.302585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51933bcf-8160-481f-9c99-89493ea8f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:28.303233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51933bcf-8160-481f-9c99-89493ea8f182 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.303513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51933bcf-8160-481f-9c99-89493ea8f182 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.303787 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51933bcf-8160-481f-9c99-89493ea8f182 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.331408 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51933bcf-8160-481f-9c99-89493ea8f182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-51933bcf-8160-481f-9c99-89493ea8f182 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:28.336909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:28.337843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:28.338616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 119/261] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:28.346659 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51933bcf-8160-481f-9c99-89493ea8f182 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:28.488713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37088da-0a02-4483-8cb8-1b3300aef833 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:28.517807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:28.556521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51933bcf-8160-481f-9c99-89493ea8f182 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:28.562628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d9484-98ac-4c4e-92a7-0223df663b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:28.563281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d9484-98ac-4c4e-92a7-0223df663b85 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.563563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d9484-98ac-4c4e-92a7-0223df663b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.563835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d9484-98ac-4c4e-92a7-0223df663b85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.633917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e96ce-65c2-461b-9fc7-68eecd5821f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:28.634541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e96ce-65c2-461b-9fc7-68eecd5821f7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.634821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e96ce-65c2-461b-9fc7-68eecd5821f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.635090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e96ce-65c2-461b-9fc7-68eecd5821f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.635771 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e96ce-65c2-461b-9fc7-68eecd5821f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:28.636072 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e96ce-65c2-461b-9fc7-68eecd5821f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:28.637650 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e96ce-65c2-461b-9fc7-68eecd5821f7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:28.656077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 120/262] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:28.661703 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e96ce-65c2-461b-9fc7-68eecd5821f7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:28.697714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:28.704599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d9484-98ac-4c4e-92a7-0223df663b85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:28.708976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d9484-98ac-4c4e-92a7-0223df663b85 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:28.850141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 143/263] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:28.856374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d9484-98ac-4c4e-92a7-0223df663b85 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:28.877632 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:28.912280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e96ce-65c2-461b-9fc7-68eecd5821f7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:28.967976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e682dd-776e-446c-b787-b4ff18c34484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:28.969008 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e682dd-776e-446c-b787-b4ff18c34484 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.969432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e682dd-776e-446c-b787-b4ff18c34484 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.969951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e682dd-776e-446c-b787-b4ff18c34484 None admin] PATH_INFO: `/v3/users/3c8b48ee00d947abb05c01ae2488d550` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.971110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e682dd-776e-446c-b787-b4ff18c34484 None admin] RBAC: Authorizing `identity:get_user(user_id=3c8b48ee00d947abb05c01ae2488d550)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:28.972982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e682dd-776e-446c-b787-b4ff18c34484 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:28.974602 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 121/264] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:28 2022] GET /identity/v3/users/3c8b48ee00d947abb05c01ae2488d550 => generated 283 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:28.975957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb04684-da14-4263-af1e-31015120d278 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.976437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb04684-da14-4263-af1e-31015120d278 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.976933 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb04684-da14-4263-af1e-31015120d278 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.978998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 122/265] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:28.980230 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a90ed-0a89-465b-9a63-60a36e00f972 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.980818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a90ed-0a89-465b-9a63-60a36e00f972 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.985551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a90ed-0a89-465b-9a63-60a36e00f972 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.987632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 123/266] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:28.989129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191eb2c-7156-47e1-8d72-90db9f7932fb None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.989683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191eb2c-7156-47e1-8d72-90db9f7932fb None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.990076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191eb2c-7156-47e1-8d72-90db9f7932fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.992829 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d9484-98ac-4c4e-92a7-0223df663b85 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:29.014679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a191eb2c-7156-47e1-8d72-90db9f7932fb None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:29.016018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a191eb2c-7156-47e1-8d72-90db9f7932fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:29.034564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7d4a4-3d4c-4fa2-a2f5-1e0d72d2d103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:29.035465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7d4a4-3d4c-4fa2-a2f5-1e0d72d2d103 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.035884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7d4a4-3d4c-4fa2-a2f5-1e0d72d2d103 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.036299 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7d4a4-3d4c-4fa2-a2f5-1e0d72d2d103 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.038110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7d4a4-3d4c-4fa2-a2f5-1e0d72d2d103 None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:29.040166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7d4a4-3d4c-4fa2-a2f5-1e0d72d2d103 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:29.043081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 144/267] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:28 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:29.048304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7d4a4-3d4c-4fa2-a2f5-1e0d72d2d103 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:29.100394 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 124/268] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:29.102556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544628f0-0c32-4cf9-b3d8-0f2ebdbfdd4e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.102969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544628f0-0c32-4cf9-b3d8-0f2ebdbfdd4e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.103372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544628f0-0c32-4cf9-b3d8-0f2ebdbfdd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.128098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-544628f0-0c32-4cf9-b3d8-0f2ebdbfdd4e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:29.129775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-544628f0-0c32-4cf9-b3d8-0f2ebdbfdd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:29.145717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:29.148169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:29.186883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7d4a4-3d4c-4fa2-a2f5-1e0d72d2d103 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:29.250348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922821ad-2183-4c70-86cd-1f1c5fda131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:29.250959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922821ad-2183-4c70-86cd-1f1c5fda131e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.251235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922821ad-2183-4c70-86cd-1f1c5fda131e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.251499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922821ad-2183-4c70-86cd-1f1c5fda131e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.264471 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 125/269] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:29.265364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544628f0-0c32-4cf9-b3d8-0f2ebdbfdd4e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:29.352810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922821ad-2183-4c70-86cd-1f1c5fda131e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:29.354663 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922821ad-2183-4c70-86cd-1f1c5fda131e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:29.436792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:29.469452 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 145/270] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:29.471066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a7e39-6fbc-4b37-8484-01bc31b7b841 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.472947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a7e39-6fbc-4b37-8484-01bc31b7b841 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.473235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a7e39-6fbc-4b37-8484-01bc31b7b841 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.476734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 146/271] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:29.477500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a7e39-6fbc-4b37-8484-01bc31b7b841 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:29.488933 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544628f0-0c32-4cf9-b3d8-0f2ebdbfdd4e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:29.532920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed28987-ad4a-4111-817c-82e34d53dda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:29.533822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed28987-ad4a-4111-817c-82e34d53dda8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.534102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed28987-ad4a-4111-817c-82e34d53dda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.534369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed28987-ad4a-4111-817c-82e34d53dda8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.535102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed28987-ad4a-4111-817c-82e34d53dda8 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=9549a5efb25e493d9a9b245fcc802166, user.id=3c8b48ee00d947abb05c01ae2488d550) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:29.535561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed28987-ad4a-4111-817c-82e34d53dda8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:29.537657 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed28987-ad4a-4111-817c-82e34d53dda8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:29.544906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 126/272] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:29 2022] GET /identity/v3/role_assignments?user.id=3c8b48ee00d947abb05c01ae2488d550&scope.project.id=9549a5efb25e493d9a9b245fcc802166&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:29.545751 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed28987-ad4a-4111-817c-82e34d53dda8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:29.577273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:29.615220 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a7e39-6fbc-4b37-8484-01bc31b7b841 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:29.653567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99acea-fedb-41a0-b14e-b8f01aa974f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:29.654241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99acea-fedb-41a0-b14e-b8f01aa974f8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.654564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99acea-fedb-41a0-b14e-b8f01aa974f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.654838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99acea-fedb-41a0-b14e-b8f01aa974f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.666824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99acea-fedb-41a0-b14e-b8f01aa974f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-8b99acea-fedb-41a0-b14e-b8f01aa974f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:29.672226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:29.673195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:29.673977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 147/273] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:29.674843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff9e43-8d4a-409c-9be0-751a51b74d3f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.675091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff9e43-8d4a-409c-9be0-751a51b74d3f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.675367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff9e43-8d4a-409c-9be0-751a51b74d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.697646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a1ff9e43-8d4a-409c-9be0-751a51b74d3f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:29.699087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a1ff9e43-8d4a-409c-9be0-751a51b74d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:29.699204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed28987-ad4a-4111-817c-82e34d53dda8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:29.739842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332eab4-a92c-4b7c-98de-3c480d2a551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:29.740483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332eab4-a92c-4b7c-98de-3c480d2a551c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.741015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:29.741134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332eab4-a92c-4b7c-98de-3c480d2a551c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.741134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332eab4-a92c-4b7c-98de-3c480d2a551c None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.742193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332eab4-a92c-4b7c-98de-3c480d2a551c None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:29.743956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332eab4-a92c-4b7c-98de-3c480d2a551c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:29.745699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 127/274] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:29 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:29.747540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332eab4-a92c-4b7c-98de-3c480d2a551c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:29.785675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 148/275] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:29.786534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff9e43-8d4a-409c-9be0-751a51b74d3f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:29.860436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:29.884729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332eab4-a92c-4b7c-98de-3c480d2a551c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:29.905862 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff9e43-8d4a-409c-9be0-751a51b74d3f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:29.924417 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc07c3-833f-4949-9c72-567b75817fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:29.925858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc07c3-833f-4949-9c72-567b75817fae None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.926391 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc07c3-833f-4949-9c72-567b75817fae None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.926848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc07c3-833f-4949-9c72-567b75817fae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.939220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc07c3-833f-4949-9c72-567b75817fae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:29.940160 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38662e04-ca60-462b-abd9-f9a905b3de13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:29.940888 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38662e04-ca60-462b-abd9-f9a905b3de13 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.941221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38662e04-ca60-462b-abd9-f9a905b3de13 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.941557 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38662e04-ca60-462b-abd9-f9a905b3de13 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.942335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38662e04-ca60-462b-abd9-f9a905b3de13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:29.942694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38662e04-ca60-462b-abd9-f9a905b3de13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:29.944192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38662e04-ca60-462b-abd9-f9a905b3de13 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-cabc07c3-833f-4949-9c72-567b75817fae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:29.946652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:29.947504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:29.948283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:29.948283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:29.948283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:29.948283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:29.948283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:29.948283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:29.948283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:29.948283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:29.948283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:29.948283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:29.948283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:29.948283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:29.951355 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 128/276] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:29.952457 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc07c3-833f-4949-9c72-567b75817fae None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:29.953648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 149/277] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:29.959914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38662e04-ca60-462b-abd9-f9a905b3de13 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:29.999093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:30.077979 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38662e04-ca60-462b-abd9-f9a905b3de13 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:30.090173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc07c3-833f-4949-9c72-567b75817fae None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:30.113714 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd286d-e63a-40a6-85db-3a633bf0983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:30.114313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd286d-e63a-40a6-85db-3a633bf0983d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.114586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd286d-e63a-40a6-85db-3a633bf0983d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.114851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd286d-e63a-40a6-85db-3a633bf0983d None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.115505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd286d-e63a-40a6-85db-3a633bf0983d None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:30.115801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd286d-e63a-40a6-85db-3a633bf0983d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:30.117209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd286d-e63a-40a6-85db-3a633bf0983d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:30.124422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4c74f-57c0-4b5c-9bc0-465b1798dc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:30.125059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4c74f-57c0-4b5c-9bc0-465b1798dc46 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.125335 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4c74f-57c0-4b5c-9bc0-465b1798dc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.125609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4c74f-57c0-4b5c-9bc0-465b1798dc46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.125798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 150/278] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:30.126399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4c74f-57c0-4b5c-9bc0-465b1798dc46 None admin] RBAC: Adding query filter params (role.id=16317a22e9c94a8886f02efb75ab456e, scope.project.id=9549a5efb25e493d9a9b245fcc802166, user.id=3c8b48ee00d947abb05c01ae2488d550) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:30.126632 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd286d-e63a-40a6-85db-3a633bf0983d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:30.126801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4c74f-57c0-4b5c-9bc0-465b1798dc46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:30.128248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4c74f-57c0-4b5c-9bc0-465b1798dc46 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:30.134197 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 129/279] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:29 2022] GET /identity/v3/role_assignments?user.id=3c8b48ee00d947abb05c01ae2488d550&scope.project.id=9549a5efb25e493d9a9b245fcc802166&role.id=16317a22e9c94a8886f02efb75ab456e => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:30.135461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbf025-8d70-485c-ab40-d4d72d9c31af None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.135694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbf025-8d70-485c-ab40-d4d72d9c31af None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.135966 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbf025-8d70-485c-ab40-d4d72d9c31af None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.140746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 130/280] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:30.140746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbf025-8d70-485c-ab40-d4d72d9c31af None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:30.144191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:30.244980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:30.253391 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd286d-e63a-40a6-85db-3a633bf0983d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:30.258998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbf025-8d70-485c-ab40-d4d72d9c31af None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:30.295080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac59d2-cda3-4d9c-9d1e-af51df77950d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:30.295722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac59d2-cda3-4d9c-9d1e-af51df77950d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.296002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac59d2-cda3-4d9c-9d1e-af51df77950d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.296277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac59d2-cda3-4d9c-9d1e-af51df77950d None admin] PATH_INFO: `/v3/users/7312047a0e204706b67cec86de5fa181` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.297448 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac59d2-cda3-4d9c-9d1e-af51df77950d None admin] RBAC: Authorizing `identity:get_user(user_id=7312047a0e204706b67cec86de5fa181)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:30.298362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9a5b5-de4e-4198-be12-99f067794f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:30.298908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac59d2-cda3-4d9c-9d1e-af51df77950d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:30.299009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9a5b5-de4e-4198-be12-99f067794f13 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.299264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9a5b5-de4e-4198-be12-99f067794f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.299534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9a5b5-de4e-4198-be12-99f067794f13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.300518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 151/281] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:30 2022] GET /identity/v3/users/7312047a0e204706b67cec86de5fa181 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:30.300787 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9a5b5-de4e-4198-be12-99f067794f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:30.301319 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac59d2-cda3-4d9c-9d1e-af51df77950d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:30.302182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9a5b5-de4e-4198-be12-99f067794f13 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:30.304075 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 131/282] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:30.306366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1b0f4-3dba-4e9a-addc-5489fa1b58e9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.306645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1b0f4-3dba-4e9a-addc-5489fa1b58e9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.306919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1b0f4-3dba-4e9a-addc-5489fa1b58e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.325499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-12f1b0f4-3dba-4e9a-addc-5489fa1b58e9 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:30.326939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-12f1b0f4-3dba-4e9a-addc-5489fa1b58e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:30.397765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:30.402818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 132/283] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:30.403879 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1b0f4-3dba-4e9a-addc-5489fa1b58e9 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:30.411377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac59d2-cda3-4d9c-9d1e-af51df77950d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:30.447323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8130c32-749b-405f-bb51-85e97aecd7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:30.447956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8130c32-749b-405f-bb51-85e97aecd7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.448237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8130c32-749b-405f-bb51-85e97aecd7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.448511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8130c32-749b-405f-bb51-85e97aecd7c8 None admin] PATH_INFO: `/v3/users/9f4b6dc33af0498bae9003eb87db2e70` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.463797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8130c32-749b-405f-bb51-85e97aecd7c8 None admin] RBAC: Authorizing `identity:get_user(user_id=9f4b6dc33af0498bae9003eb87db2e70)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:30.465767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8130c32-749b-405f-bb51-85e97aecd7c8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:30.467779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 152/284] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:30 2022] GET /identity/v3/users/9f4b6dc33af0498bae9003eb87db2e70 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:30.468602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8130c32-749b-405f-bb51-85e97aecd7c8 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:30.497652 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:30.511880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1b0f4-3dba-4e9a-addc-5489fa1b58e9 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:30.548189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b16d0d-5d5f-456f-ac6b-1943b1c8f7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:30.548845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b16d0d-5d5f-456f-ac6b-1943b1c8f7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.549218 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b16d0d-5d5f-456f-ac6b-1943b1c8f7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.549485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b16d0d-5d5f-456f-ac6b-1943b1c8f7c4 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.550611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b16d0d-5d5f-456f-ac6b-1943b1c8f7c4 None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:30.552065 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b16d0d-5d5f-456f-ac6b-1943b1c8f7c4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:30.558518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 133/285] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:30 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:30.558518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b16d0d-5d5f-456f-ac6b-1943b1c8f7c4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:30.584491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8130c32-749b-405f-bb51-85e97aecd7c8 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:30.626512 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2162c0-2d7f-489e-ad36-18808fa4fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:30.627116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2162c0-2d7f-489e-ad36-18808fa4fd63 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.627388 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2162c0-2d7f-489e-ad36-18808fa4fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.627653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2162c0-2d7f-489e-ad36-18808fa4fd63 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.628297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2162c0-2d7f-489e-ad36-18808fa4fd63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:30.629670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:30.629777 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2162c0-2d7f-489e-ad36-18808fa4fd63 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:30.655180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 153/286] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:30 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:30.656331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2162c0-2d7f-489e-ad36-18808fa4fd63 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:30.671948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b16d0d-5d5f-456f-ac6b-1943b1c8f7c4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:30.711964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17fcf4-5574-4b91-9431-8758c1fc3475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:30.712578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17fcf4-5574-4b91-9431-8758c1fc3475 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.712871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17fcf4-5574-4b91-9431-8758c1fc3475 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.713140 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17fcf4-5574-4b91-9431-8758c1fc3475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.714192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17fcf4-5574-4b91-9431-8758c1fc3475 None admin] RBAC: Adding query filter params (role.id=3647def0715f4fd4991f82b06815ee38, user.id=9f4b6dc33af0498bae9003eb87db2e70, scope.system=all) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:30.714493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17fcf4-5574-4b91-9431-8758c1fc3475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:30.716360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17fcf4-5574-4b91-9431-8758c1fc3475 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:30.722545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 134/287] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Aug 24 12:19:30 2022] GET /identity/v3/role_assignments?user.id=9f4b6dc33af0498bae9003eb87db2e70&scope.system=all&role.id=3647def0715f4fd4991f82b06815ee38 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:30.729746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17fcf4-5574-4b91-9431-8758c1fc3475 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:30.763221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:30.792756 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2162c0-2d7f-489e-ad36-18808fa4fd63 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:30.849402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e760b-64d0-4f46-ac6d-dd0b11897c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:30.850086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e760b-64d0-4f46-ac6d-dd0b11897c49 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.850420 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e760b-64d0-4f46-ac6d-dd0b11897c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.850683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e760b-64d0-4f46-ac6d-dd0b11897c49 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166/users/7312047a0e204706b67cec86de5fa181/roles/26063cb085b6415989a19c8e7dc2c3c5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.852737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e760b-64d0-4f46-ac6d-dd0b11897c49 None admin] RBAC: Authorizing `identity:create_grant(project_id=9549a5efb25e493d9a9b245fcc802166, user_id=7312047a0e204706b67cec86de5fa181, role_id=26063cb085b6415989a19c8e7dc2c3c5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:30.855061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e760b-64d0-4f46-ac6d-dd0b11897c49 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:30.873873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 154/288] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:30 2022] PUT /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166/users/7312047a0e204706b67cec86de5fa181/roles/26063cb085b6415989a19c8e7dc2c3c5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:30.894581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17fcf4-5574-4b91-9431-8758c1fc3475 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:30.930756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:30.939053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e6847-dd60-4791-a092-c99a68e5170e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:30.939691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e6847-dd60-4791-a092-c99a68e5170e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.939959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e6847-dd60-4791-a092-c99a68e5170e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.940226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e6847-dd60-4791-a092-c99a68e5170e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.953257 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e6847-dd60-4791-a092-c99a68e5170e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-222e6847-dd60-4791-a092-c99a68e5170e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:30.960301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:30.961315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:30.962100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 135/289] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:30.968451 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e760b-64d0-4f46-ac6d-dd0b11897c49 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:31.090486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e760b-64d0-4f46-ac6d-dd0b11897c49 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:31.126304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7c281-ce26-4e74-b3ab-350284d97a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:31.127170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7c281-ce26-4e74-b3ab-350284d97a76 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:31.127581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7c281-ce26-4e74-b3ab-350284d97a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:31.128016 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7c281-ce26-4e74-b3ab-350284d97a76 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:31.128855 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7c281-ce26-4e74-b3ab-350284d97a76 None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:31.129289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7c281-ce26-4e74-b3ab-350284d97a76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:31.130802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7c281-ce26-4e74-b3ab-350284d97a76 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:31.136962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:31.142551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 155/290] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:31.148258 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e6847-dd60-4791-a092-c99a68e5170e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:31.263341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e6847-dd60-4791-a092-c99a68e5170e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:31.308258 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55113523-7c8c-4948-b0f5-e3125fa87860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:31.309057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55113523-7c8c-4948-b0f5-e3125fa87860 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:31.309479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55113523-7c8c-4948-b0f5-e3125fa87860 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:31.309967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55113523-7c8c-4948-b0f5-e3125fa87860 None admin] PATH_INFO: `/v3/users/dcd2ab21c0804e658b41f9945b48667f` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:31.311226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55113523-7c8c-4948-b0f5-e3125fa87860 None admin] RBAC: Authorizing `identity:get_user(user_id=dcd2ab21c0804e658b41f9945b48667f)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:31.312750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55113523-7c8c-4948-b0f5-e3125fa87860 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:31.314385 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 136/291] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:31 2022] GET /identity/v3/users/dcd2ab21c0804e658b41f9945b48667f => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:31.320315 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7c281-ce26-4e74-b3ab-350284d97a76 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:31.495106 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7c281-ce26-4e74-b3ab-350284d97a76 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:31.536218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311c054-0a90-4692-aa8b-178717580c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:31.537136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311c054-0a90-4692-aa8b-178717580c77 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:31.537581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311c054-0a90-4692-aa8b-178717580c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:31.538043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311c054-0a90-4692-aa8b-178717580c77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:31.613858 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:31.616946 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311c054-0a90-4692-aa8b-178717580c77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:31.618624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311c054-0a90-4692-aa8b-178717580c77 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:31.735852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 156/292] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:31.741314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55113523-7c8c-4948-b0f5-e3125fa87860 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:31.817244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:31.875611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55113523-7c8c-4948-b0f5-e3125fa87860 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:31.927225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfee3f-9697-433b-843c-48c502ecfd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:31.927867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfee3f-9697-433b-843c-48c502ecfd6d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:31.928147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfee3f-9697-433b-843c-48c502ecfd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:31.928421 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfee3f-9697-433b-843c-48c502ecfd6d None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:31.929648 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bfee3f-9697-433b-843c-48c502ecfd6d None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:31.931140 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bfee3f-9697-433b-843c-48c502ecfd6d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:31.932817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 137/293] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:31 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:31.939363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311c054-0a90-4692-aa8b-178717580c77 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:32.079409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:32.082614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311c054-0a90-4692-aa8b-178717580c77 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:32.128707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb17fc-8485-421d-8222-c477315173fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:32.129330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb17fc-8485-421d-8222-c477315173fe None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.129589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb17fc-8485-421d-8222-c477315173fe None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.129856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb17fc-8485-421d-8222-c477315173fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.130587 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb17fc-8485-421d-8222-c477315173fe None admin] RBAC: Adding query filter params (role.id=3647def0715f4fd4991f82b06815ee38, scope.project.id=9549a5efb25e493d9a9b245fcc802166, user.id=dcd2ab21c0804e658b41f9945b48667f) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:32.130892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb17fc-8485-421d-8222-c477315173fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:32.132333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb17fc-8485-421d-8222-c477315173fe None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:32.139575 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 157/294] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:31 2022] GET /identity/v3/role_assignments?user.id=dcd2ab21c0804e658b41f9945b48667f&scope.project.id=9549a5efb25e493d9a9b245fcc802166&role.id=3647def0715f4fd4991f82b06815ee38 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:32.772997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c4eb7-80a0-4b31-a374-ec1fcb657034 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.773693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c4eb7-80a0-4b31-a374-ec1fcb657034 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.774166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c4eb7-80a0-4b31-a374-ec1fcb657034 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.775941 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 138/295] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:32.782077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82a9b2-6709-43e0-870d-3bc6b9e902c9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.782688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82a9b2-6709-43e0-870d-3bc6b9e902c9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.783151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82a9b2-6709-43e0-870d-3bc6b9e902c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.805161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6d82a9b2-6709-43e0-870d-3bc6b9e902c9 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:32.806734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6d82a9b2-6709-43e0-870d-3bc6b9e902c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:32.885994 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 158/296] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:33.065062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2a400-88af-47ee-b581-b097fa9a1447 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.065551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2a400-88af-47ee-b581-b097fa9a1447 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.065551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2a400-88af-47ee-b581-b097fa9a1447 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.067151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 139/297] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:33.072458 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1fa76-b844-4c6a-8b95-6a93ad92dcc7 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.072784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1fa76-b844-4c6a-8b95-6a93ad92dcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.073062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1fa76-b844-4c6a-8b95-6a93ad92dcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.091881 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9af1fa76-b844-4c6a-8b95-6a93ad92dcc7 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:33.093090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9af1fa76-b844-4c6a-8b95-6a93ad92dcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:33.146545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9acde-ee29-45ba-a5fe-d54476f1a221 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.146851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9acde-ee29-45ba-a5fe-d54476f1a221 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.147082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9acde-ee29-45ba-a5fe-d54476f1a221 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.148437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 140/298] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:33.153977 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1376b5-2b23-40be-8de4-e5ce3bcd1293 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.154507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1376b5-2b23-40be-8de4-e5ce3bcd1293 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.154943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1376b5-2b23-40be-8de4-e5ce3bcd1293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.164334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 159/299] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:33.167606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1fa76-b844-4c6a-8b95-6a93ad92dcc7 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.176289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-fa1376b5-2b23-40be-8de4-e5ce3bcd1293 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:33.178205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fa1376b5-2b23-40be-8de4-e5ce3bcd1293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:33.193676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:33.270556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 141/300] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:33.280297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1fa76-b844-4c6a-8b95-6a93ad92dcc7 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.315006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c10c7-272f-4df5-805b-40967216b349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.315703 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c10c7-272f-4df5-805b-40967216b349 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.315918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c10c7-272f-4df5-805b-40967216b349 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.316185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c10c7-272f-4df5-805b-40967216b349 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.327289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c10c7-272f-4df5-805b-40967216b349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-501c10c7-272f-4df5-805b-40967216b349 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:33.332116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:33.333184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:33.333983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 160/301] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:33.341080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1376b5-2b23-40be-8de4-e5ce3bcd1293 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.402063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:33.439917 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c10c7-272f-4df5-805b-40967216b349 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.445719 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1376b5-2b23-40be-8de4-e5ce3bcd1293 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.481539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f20e31-e4d5-4e96-9c5e-fb40a2208fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.482256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f20e31-e4d5-4e96-9c5e-fb40a2208fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.482532 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f20e31-e4d5-4e96-9c5e-fb40a2208fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.482796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f20e31-e4d5-4e96-9c5e-fb40a2208fb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.483574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f20e31-e4d5-4e96-9c5e-fb40a2208fb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:33.483880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f20e31-e4d5-4e96-9c5e-fb40a2208fb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:33.485288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f20e31-e4d5-4e96-9c5e-fb40a2208fb3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:33.496629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 142/302] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:33.502168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f20e31-e4d5-4e96-9c5e-fb40a2208fb3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.541210 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:33.558054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c10c7-272f-4df5-805b-40967216b349 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.593273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f8dac-6256-41f3-8a59-a759cc4ba191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.593820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8dac-6256-41f3-8a59-a759cc4ba191 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.594094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8dac-6256-41f3-8a59-a759cc4ba191 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.594361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8dac-6256-41f3-8a59-a759cc4ba191 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.604321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f8dac-6256-41f3-8a59-a759cc4ba191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:33.608208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f20e31-e4d5-4e96-9c5e-fb40a2208fb3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-456f8dac-6256-41f3-8a59-a759cc4ba191 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:33.609177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:33.610079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:33.610884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:33.610884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:33.610884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:33.610884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:33.610884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:33.610884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:33.610884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:33.610884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:33.610884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:33.610884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:33.610884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:33.610884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:33.612026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 161/303] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:33.613380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3203736-4754-42fa-96cf-aff91f2a8a57 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.613660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3203736-4754-42fa-96cf-aff91f2a8a57 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.613931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3203736-4754-42fa-96cf-aff91f2a8a57 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.615283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 162/304] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:33.616020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3203736-4754-42fa-96cf-aff91f2a8a57 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.628746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:33.643963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13cc9e-4f0c-417f-98f4-3506fa4abd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.644596 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13cc9e-4f0c-417f-98f4-3506fa4abd7c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.644903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13cc9e-4f0c-417f-98f4-3506fa4abd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.645177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13cc9e-4f0c-417f-98f4-3506fa4abd7c None admin] PATH_INFO: `/v3/users/9f4b6dc33af0498bae9003eb87db2e70` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.646318 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13cc9e-4f0c-417f-98f4-3506fa4abd7c None admin] RBAC: Authorizing `identity:get_user(user_id=9f4b6dc33af0498bae9003eb87db2e70)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:33.647847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13cc9e-4f0c-417f-98f4-3506fa4abd7c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:33.654599 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 143/305] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/users/9f4b6dc33af0498bae9003eb87db2e70 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:33.654599 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13cc9e-4f0c-417f-98f4-3506fa4abd7c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.720461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3203736-4754-42fa-96cf-aff91f2a8a57 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.753525 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c650191-0269-4cfa-aaa0-b25a8e59fa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.754148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c650191-0269-4cfa-aaa0-b25a8e59fa1f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.754418 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c650191-0269-4cfa-aaa0-b25a8e59fa1f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.754712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c650191-0269-4cfa-aaa0-b25a8e59fa1f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.754992 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13cc9e-4f0c-417f-98f4-3506fa4abd7c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.765049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c650191-0269-4cfa-aaa0-b25a8e59fa1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-9c650191-0269-4cfa-aaa0-b25a8e59fa1f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:33.769790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:33.770814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:33.771631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 163/306] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:33.772255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08243a-1233-464d-8ae8-87b07e08a5d2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.772524 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08243a-1233-464d-8ae8-87b07e08a5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.772675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08243a-1233-464d-8ae8-87b07e08a5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.780283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:33.791126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-be08243a-1233-464d-8ae8-87b07e08a5d2 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:33.792308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-be08243a-1233-464d-8ae8-87b07e08a5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:33.793845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0454f-0d58-4043-a4a4-57403fd61ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.794636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0454f-0d58-4043-a4a4-57403fd61ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.794910 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0454f-0d58-4043-a4a4-57403fd61ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.795176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0454f-0d58-4043-a4a4-57403fd61ad1 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.795833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0454f-0d58-4043-a4a4-57403fd61ad1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:33.796138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0454f-0d58-4043-a4a4-57403fd61ad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:33.797531 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0454f-0d58-4043-a4a4-57403fd61ad1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:33.805745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 144/307] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:33.806526 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0454f-0d58-4043-a4a4-57403fd61ad1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.857554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:33.860469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 164/308] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:33.861356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08243a-1233-464d-8ae8-87b07e08a5d2 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.914792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0454f-0d58-4043-a4a4-57403fd61ad1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.951420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7310c-9223-4256-a455-097d693ba945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.952071 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7310c-9223-4256-a455-097d693ba945 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.952348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7310c-9223-4256-a455-097d693ba945 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.952616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7310c-9223-4256-a455-097d693ba945 None admin] PATH_INFO: `/v3/system/users/9f4b6dc33af0498bae9003eb87db2e70/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.954070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7310c-9223-4256-a455-097d693ba945 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9f4b6dc33af0498bae9003eb87db2e70, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:33.955510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c7310c-9223-4256-a455-097d693ba945 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:33.966241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 145/309] 10.222.0.27 () {58 vars in 1334 bytes} [Wed Aug 24 12:19:33 2022] PUT /identity/v3/system/users/9f4b6dc33af0498bae9003eb87db2e70/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:33.967482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7310c-9223-4256-a455-097d693ba945 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.967981 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08243a-1233-464d-8ae8-87b07e08a5d2 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.032578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc3df4-8bdc-4364-ab12-6dde55056fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.033884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3df4-8bdc-4364-ab12-6dde55056fbc None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.034456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3df4-8bdc-4364-ab12-6dde55056fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.034968 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3df4-8bdc-4364-ab12-6dde55056fbc None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.035924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc3df4-8bdc-4364-ab12-6dde55056fbc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:34.036423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc3df4-8bdc-4364-ab12-6dde55056fbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.038321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc3df4-8bdc-4364-ab12-6dde55056fbc None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.040474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:34.052475 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 165/310] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:34.058059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc3df4-8bdc-4364-ab12-6dde55056fbc None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.104758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7310c-9223-4256-a455-097d693ba945 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.122286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:34.148847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d936da5-f568-4858-914f-dbddc41128c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.149648 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d936da5-f568-4858-914f-dbddc41128c6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.149988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d936da5-f568-4858-914f-dbddc41128c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.150259 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d936da5-f568-4858-914f-dbddc41128c6 None admin] PATH_INFO: `/v3/users/2b3d4c7096134887a00e916ce00a0ee7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.165671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d936da5-f568-4858-914f-dbddc41128c6 None admin] RBAC: Authorizing `identity:get_user(user_id=2b3d4c7096134887a00e916ce00a0ee7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.167246 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d936da5-f568-4858-914f-dbddc41128c6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.172987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 146/311] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/users/2b3d4c7096134887a00e916ce00a0ee7 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:34.172987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d936da5-f568-4858-914f-dbddc41128c6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.188006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc3df4-8bdc-4364-ab12-6dde55056fbc None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.221469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80259f-adac-4ee0-8a3e-7d7707d8d0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.222097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80259f-adac-4ee0-8a3e-7d7707d8d0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.222370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80259f-adac-4ee0-8a3e-7d7707d8d0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.222633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80259f-adac-4ee0-8a3e-7d7707d8d0f2 None admin] PATH_INFO: `/v3/users/7312047a0e204706b67cec86de5fa181` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.224186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80259f-adac-4ee0-8a3e-7d7707d8d0f2 None admin] RBAC: Authorizing `identity:get_user(user_id=7312047a0e204706b67cec86de5fa181)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.225663 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80259f-adac-4ee0-8a3e-7d7707d8d0f2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.227242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 166/312] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:34 2022] GET /identity/v3/users/7312047a0e204706b67cec86de5fa181 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:34.227990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80259f-adac-4ee0-8a3e-7d7707d8d0f2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.280598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d936da5-f568-4858-914f-dbddc41128c6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.293782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:34.319930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02d3d3-233d-4dc5-8716-4494e4e9f4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.320934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02d3d3-233d-4dc5-8716-4494e4e9f4ba None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.321232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02d3d3-233d-4dc5-8716-4494e4e9f4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.321517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02d3d3-233d-4dc5-8716-4494e4e9f4ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.322636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02d3d3-233d-4dc5-8716-4494e4e9f4ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.324236 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02d3d3-233d-4dc5-8716-4494e4e9f4ba None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.326239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 147/313] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:34.326239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02d3d3-233d-4dc5-8716-4494e4e9f4ba None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.341720 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80259f-adac-4ee0-8a3e-7d7707d8d0f2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.374741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:34.376357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb064fa6-f612-448c-98b0-09828e8caee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.377004 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb064fa6-f612-448c-98b0-09828e8caee6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.377286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb064fa6-f612-448c-98b0-09828e8caee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.377560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb064fa6-f612-448c-98b0-09828e8caee6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.378500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb064fa6-f612-448c-98b0-09828e8caee6 None admin] RBAC: Adding query filter params (role.id=26063cb085b6415989a19c8e7dc2c3c5, user.id=2b3d4c7096134887a00e916ce00a0ee7, scope.system=all) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:34.378811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb064fa6-f612-448c-98b0-09828e8caee6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.380268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb064fa6-f612-448c-98b0-09828e8caee6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.385659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 167/314] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Aug 24 12:19:34 2022] GET /identity/v3/role_assignments?user.id=2b3d4c7096134887a00e916ce00a0ee7&scope.system=all&role.id=26063cb085b6415989a19c8e7dc2c3c5 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:34.386441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb064fa6-f612-448c-98b0-09828e8caee6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.457156 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02d3d3-233d-4dc5-8716-4494e4e9f4ba None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.504975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb708c84-b938-4be8-b1f7-0a342289952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.505599 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb708c84-b938-4be8-b1f7-0a342289952d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.505878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb708c84-b938-4be8-b1f7-0a342289952d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.506149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb708c84-b938-4be8-b1f7-0a342289952d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.523902 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb064fa6-f612-448c-98b0-09828e8caee6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.559687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:34.569363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c95d7-906f-4096-b7d5-492d7e9fab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.569978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c95d7-906f-4096-b7d5-492d7e9fab6c None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.570256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c95d7-906f-4096-b7d5-492d7e9fab6c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.570527 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c95d7-906f-4096-b7d5-492d7e9fab6c None admin] PATH_INFO: `/v3/groups` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.571157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c95d7-906f-4096-b7d5-492d7e9fab6c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.572603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c95d7-906f-4096-b7d5-492d7e9fab6c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.595009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 168/315] 10.222.0.27 () {60 vars in 1134 bytes} [Wed Aug 24 12:19:34 2022] POST /identity/v3/groups => generated 229 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:34.599575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb708c84-b938-4be8-b1f7-0a342289952d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.604717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb708c84-b938-4be8-b1f7-0a342289952d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.654400 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:34.729510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 148/316] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:34.735531 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c95d7-906f-4096-b7d5-492d7e9fab6c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.844602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c95d7-906f-4096-b7d5-492d7e9fab6c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.879061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e32ad-8f0f-4a58-8829-f6ff72d9810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.879697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e32ad-8f0f-4a58-8829-f6ff72d9810a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.879993 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e32ad-8f0f-4a58-8829-f6ff72d9810a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.880237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e32ad-8f0f-4a58-8829-f6ff72d9810a None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.881368 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e32ad-8f0f-4a58-8829-f6ff72d9810a None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.882820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e32ad-8f0f-4a58-8829-f6ff72d9810a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.884765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 169/317] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:34 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:34.891636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb708c84-b938-4be8-b1f7-0a342289952d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:35.006956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb708c84-b938-4be8-b1f7-0a342289952d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:35.044154 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f39fc-9c65-44c4-9fd6-ad7d0cec2c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:35.044789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f39fc-9c65-44c4-9fd6-ad7d0cec2c25 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:35.045070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f39fc-9c65-44c4-9fd6-ad7d0cec2c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:35.045337 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f39fc-9c65-44c4-9fd6-ad7d0cec2c25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:35.046078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f39fc-9c65-44c4-9fd6-ad7d0cec2c25 None admin] RBAC: Adding query filter params (role.id=26063cb085b6415989a19c8e7dc2c3c5, scope.project.id=9549a5efb25e493d9a9b245fcc802166, user.id=7312047a0e204706b67cec86de5fa181) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:35.046382 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f39fc-9c65-44c4-9fd6-ad7d0cec2c25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:35.047842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f39fc-9c65-44c4-9fd6-ad7d0cec2c25 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:35.054687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 149/318] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:34 2022] GET /identity/v3/role_assignments?user.id=7312047a0e204706b67cec86de5fa181&scope.project.id=9549a5efb25e493d9a9b245fcc802166&role.id=26063cb085b6415989a19c8e7dc2c3c5 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:35.915048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0a219-bfca-4e37-85c5-70a4f64850e0 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:35.915588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0a219-bfca-4e37-85c5-70a4f64850e0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:35.915588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0a219-bfca-4e37-85c5-70a4f64850e0 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:35.917142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 170/319] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:35.922743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912571e-037b-4fae-867d-9f64a0fcd27a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:35.923163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912571e-037b-4fae-867d-9f64a0fcd27a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:35.923334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912571e-037b-4fae-867d-9f64a0fcd27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:35.930823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:35.943083 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c912571e-037b-4fae-867d-9f64a0fcd27a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:35.944284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c912571e-037b-4fae-867d-9f64a0fcd27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:36.019133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 150/320] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:36.289168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0a219-bfca-4e37-85c5-70a4f64850e0 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:36.307118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84931209-d423-4faa-b3b3-ab83d0c26443 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.307465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84931209-d423-4faa-b3b3-ab83d0c26443 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.307672 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84931209-d423-4faa-b3b3-ab83d0c26443 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.309315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 151/321] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:36.314378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d53d7-c8b9-4ce9-a8fd-ead21027c67a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.314660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d53d7-c8b9-4ce9-a8fd-ead21027c67a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.314930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d53d7-c8b9-4ce9-a8fd-ead21027c67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.334245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ff4d53d7-c8b9-4ce9-a8fd-ead21027c67a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:36.335464 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ff4d53d7-c8b9-4ce9-a8fd-ead21027c67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:36.354161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:36.409369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0a219-bfca-4e37-85c5-70a4f64850e0 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:36.410107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 152/322] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:36.443878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117be891-9607-43b7-9cd4-7fd5093def3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:36.444972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117be891-9607-43b7-9cd4-7fd5093def3b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.445433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117be891-9607-43b7-9cd4-7fd5093def3b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.445682 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117be891-9607-43b7-9cd4-7fd5093def3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.457616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117be891-9607-43b7-9cd4-7fd5093def3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-117be891-9607-43b7-9cd4-7fd5093def3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:36.462772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:36.463662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:36.464792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 171/323] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:36.471565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d53d7-c8b9-4ce9-a8fd-ead21027c67a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:36.536518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:36.578991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d53d7-c8b9-4ce9-a8fd-ead21027c67a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:36.615631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa3dac-162d-4183-ab91-c07d4a1ed44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:36.615952 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa3dac-162d-4183-ab91-c07d4a1ed44d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.616214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa3dac-162d-4183-ab91-c07d4a1ed44d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.616477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa3dac-162d-4183-ab91-c07d4a1ed44d None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.617140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcbdbe-8707-4ee8-936b-1242eb6a752e None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.617243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa3dac-162d-4183-ab91-c07d4a1ed44d None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:36.617414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcbdbe-8707-4ee8-936b-1242eb6a752e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.617526 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa3dac-162d-4183-ab91-c07d4a1ed44d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:36.617693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcbdbe-8707-4ee8-936b-1242eb6a752e None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.618830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa3dac-162d-4183-ab91-c07d4a1ed44d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:36.619182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 172/324] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:36.624132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406e09d-ad3f-449e-96e4-e5e6ccae278d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.624403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406e09d-ad3f-449e-96e4-e5e6ccae278d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.625269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406e09d-ad3f-449e-96e4-e5e6ccae278d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.627810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 153/325] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:36.634460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa3dac-162d-4183-ab91-c07d4a1ed44d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:36.643923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4406e09d-ad3f-449e-96e4-e5e6ccae278d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:36.645144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4406e09d-ad3f-449e-96e4-e5e6ccae278d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:36.713656 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 173/326] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:36.715391 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406e09d-ad3f-449e-96e4-e5e6ccae278d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:36.742056 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa3dac-162d-4183-ab91-c07d4a1ed44d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:36.761030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:36.773051 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:36.787108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9777dec-5241-4634-8b39-3f8e7dc1c47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:36.787271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9777dec-5241-4634-8b39-3f8e7dc1c47b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.787522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9777dec-5241-4634-8b39-3f8e7dc1c47b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.787687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9777dec-5241-4634-8b39-3f8e7dc1c47b None admin] PATH_INFO: `/v3/users/9f4b6dc33af0498bae9003eb87db2e70` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.788870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9777dec-5241-4634-8b39-3f8e7dc1c47b None admin] RBAC: Authorizing `identity:get_user(user_id=9f4b6dc33af0498bae9003eb87db2e70)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:36.790387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9777dec-5241-4634-8b39-3f8e7dc1c47b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:36.792223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 154/327] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:36 2022] GET /identity/v3/users/9f4b6dc33af0498bae9003eb87db2e70 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:36.798112 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9777dec-5241-4634-8b39-3f8e7dc1c47b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:36.818250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406e09d-ad3f-449e-96e4-e5e6ccae278d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:36.854684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90f507-ad18-43aa-8ac9-f76ed52f9026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:36.855312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90f507-ad18-43aa-8ac9-f76ed52f9026 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.855584 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90f507-ad18-43aa-8ac9-f76ed52f9026 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.856021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90f507-ad18-43aa-8ac9-f76ed52f9026 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.866519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90f507-ad18-43aa-8ac9-f76ed52f9026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3d90f507-ad18-43aa-8ac9-f76ed52f9026 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:36.871395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:36.872268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:36.873058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 174/328] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:36.878153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90f507-ad18-43aa-8ac9-f76ed52f9026 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:36.914346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9777dec-5241-4634-8b39-3f8e7dc1c47b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:36.950244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b96c54-6602-4036-a861-f9846e0c323a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:36.950867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b96c54-6602-4036-a861-f9846e0c323a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.951098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b96c54-6602-4036-a861-f9846e0c323a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.951433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b96c54-6602-4036-a861-f9846e0c323a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.952153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b96c54-6602-4036-a861-f9846e0c323a None admin] RBAC: Adding query filter params (role.id=3647def0715f4fd4991f82b06815ee38, user.id=9f4b6dc33af0498bae9003eb87db2e70, scope.system=all) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:36.952482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b96c54-6602-4036-a861-f9846e0c323a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:36.953958 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b96c54-6602-4036-a861-f9846e0c323a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:36.966486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 155/329] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Aug 24 12:19:36 2022] GET /identity/v3/role_assignments?user.id=9f4b6dc33af0498bae9003eb87db2e70&scope.system=all&role.id=3647def0715f4fd4991f82b06815ee38 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:36.985834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b96c54-6602-4036-a861-f9846e0c323a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:37.007087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90f507-ad18-43aa-8ac9-f76ed52f9026 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.016997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:37.049798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e99da-bb61-45a3-b2d3-72fc94c6fc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.050417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e99da-bb61-45a3-b2d3-72fc94c6fc15 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.050699 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e99da-bb61-45a3-b2d3-72fc94c6fc15 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.051136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e99da-bb61-45a3-b2d3-72fc94c6fc15 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.051638 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e99da-bb61-45a3-b2d3-72fc94c6fc15 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:37.051940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e99da-bb61-45a3-b2d3-72fc94c6fc15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.053382 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e99da-bb61-45a3-b2d3-72fc94c6fc15 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:37.054554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:37.063412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 175/330] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:37.069068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e99da-bb61-45a3-b2d3-72fc94c6fc15 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:37.113544 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b96c54-6602-4036-a861-f9846e0c323a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.152319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc3161-d29e-483a-b28c-7f6d15907175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.153218 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc3161-d29e-483a-b28c-7f6d15907175 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.153962 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc3161-d29e-483a-b28c-7f6d15907175 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.153962 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc3161-d29e-483a-b28c-7f6d15907175 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.155728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc3161-d29e-483a-b28c-7f6d15907175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.156476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc3161-d29e-483a-b28c-7f6d15907175 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:37.158698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 156/331] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:37.163834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc3161-d29e-483a-b28c-7f6d15907175 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:37.191067 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e99da-bb61-45a3-b2d3-72fc94c6fc15 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.223745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15572fb-fb02-4495-897b-6d63a37fb42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.224362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15572fb-fb02-4495-897b-6d63a37fb42f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.224635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15572fb-fb02-4495-897b-6d63a37fb42f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.224971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15572fb-fb02-4495-897b-6d63a37fb42f None admin] PATH_INFO: `/v3/users/2b3d4c7096134887a00e916ce00a0ee7` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.226037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15572fb-fb02-4495-897b-6d63a37fb42f None admin] RBAC: Authorizing `identity:get_user(user_id=2b3d4c7096134887a00e916ce00a0ee7)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.227472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15572fb-fb02-4495-897b-6d63a37fb42f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:37.229763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 176/332] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:37 2022] GET /identity/v3/users/2b3d4c7096134887a00e916ce00a0ee7 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:37.235355 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15572fb-fb02-4495-897b-6d63a37fb42f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:37.267889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc3161-d29e-483a-b28c-7f6d15907175 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.271383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:37.287287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:37.301825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b3498-b61a-4a22-84ea-14658ee42358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.302424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3498-b61a-4a22-84ea-14658ee42358 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.302699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3498-b61a-4a22-84ea-14658ee42358 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.302963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3498-b61a-4a22-84ea-14658ee42358 None admin] PATH_INFO: `/v3/groups` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.303577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b3498-b61a-4a22-84ea-14658ee42358 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.305047 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b3498-b61a-4a22-84ea-14658ee42358 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:37.322954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 157/333] 10.222.0.27 () {60 vars in 1134 bytes} [Wed Aug 24 12:19:37 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:19:37.337734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15572fb-fb02-4495-897b-6d63a37fb42f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.377790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920134c-2474-4f31-b0b5-cb93c6fd7725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.378374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920134c-2474-4f31-b0b5-cb93c6fd7725 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.378658 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920134c-2474-4f31-b0b5-cb93c6fd7725 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.378929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920134c-2474-4f31-b0b5-cb93c6fd7725 None admin] PATH_INFO: `/v3/system/users/2b3d4c7096134887a00e916ce00a0ee7/roles/26063cb085b6415989a19c8e7dc2c3c5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.380399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920134c-2474-4f31-b0b5-cb93c6fd7725 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2b3d4c7096134887a00e916ce00a0ee7, role_id=26063cb085b6415989a19c8e7dc2c3c5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.381849 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920134c-2474-4f31-b0b5-cb93c6fd7725 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:37.392675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 177/334] 10.222.0.27 () {58 vars in 1334 bytes} [Wed Aug 24 12:19:37 2022] PUT /identity/v3/system/users/2b3d4c7096134887a00e916ce00a0ee7/roles/26063cb085b6415989a19c8e7dc2c3c5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:19:39.601869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3479c-3ecd-42e1-a72c-2bd84429cc36 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.602384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3479c-3ecd-42e1-a72c-2bd84429cc36 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.602384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3479c-3ecd-42e1-a72c-2bd84429cc36 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.603912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 158/335] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:39.610066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a7549-53b3-4a47-a9a0-63fa914962c6 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.610533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a7549-53b3-4a47-a9a0-63fa914962c6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.610608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a7549-53b3-4a47-a9a0-63fa914962c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.637249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-da0a7549-53b3-4a47-a9a0-63fa914962c6 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:39.638491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-da0a7549-53b3-4a47-a9a0-63fa914962c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:39.654696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fe839-d67f-4cc0-9667-af8a1f021343 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.655031 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fe839-d67f-4cc0-9667-af8a1f021343 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.655232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fe839-d67f-4cc0-9667-af8a1f021343 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.656894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 159/336] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:39.662424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114353b-c57b-47a2-8480-364b0fba19bc None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.662998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114353b-c57b-47a2-8480-364b0fba19bc None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.663629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114353b-c57b-47a2-8480-364b0fba19bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.693307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d114353b-c57b-47a2-8480-364b0fba19bc None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:39.694850 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d114353b-c57b-47a2-8480-364b0fba19bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:39.765865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 178/337] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:39.813168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 160/338] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:40.096020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4577d0a-5b77-4510-8b77-2e4faae69982 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.096678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4577d0a-5b77-4510-8b77-2e4faae69982 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.097159 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4577d0a-5b77-4510-8b77-2e4faae69982 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.098978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 179/339] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:40.108039 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42b854-019c-4683-ae91-932b1ee81c4f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.108673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42b854-019c-4683-ae91-932b1ee81c4f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.109156 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42b854-019c-4683-ae91-932b1ee81c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.130841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9e42b854-019c-4683-ae91-932b1ee81c4f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:40.132360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9e42b854-019c-4683-ae91-932b1ee81c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:40.213235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 161/340] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:40.228709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4577d0a-5b77-4510-8b77-2e4faae69982 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:40.242034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a13f76-dc39-43a4-9a13-95fe9d518a06 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.244091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a13f76-dc39-43a4-9a13-95fe9d518a06 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.244572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a13f76-dc39-43a4-9a13-95fe9d518a06 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.246166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 162/341] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:40.252779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d51bb-d5b4-4dfa-879e-302f35717847 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.253330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d51bb-d5b4-4dfa-879e-302f35717847 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.253789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d51bb-d5b4-4dfa-879e-302f35717847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.291466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:40.291974 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3d1d51bb-d5b4-4dfa-879e-302f35717847 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:40.293259 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3d1d51bb-d5b4-4dfa-879e-302f35717847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:40.364387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:40.442260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 163/342] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:40.443710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d51bb-d5b4-4dfa-879e-302f35717847 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:40.458225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4577d0a-5b77-4510-8b77-2e4faae69982 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:40.540972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfbfbd-4aa5-4e31-8d66-631d724089c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:40.542003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfbfbd-4aa5-4e31-8d66-631d724089c5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.542481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfbfbd-4aa5-4e31-8d66-631d724089c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.542942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfbfbd-4aa5-4e31-8d66-631d724089c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.559031 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfbfbd-4aa5-4e31-8d66-631d724089c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-b9cfbfbd-4aa5-4e31-8d66-631d724089c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:40.565697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:40.566608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:40.567392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:40.567392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:40.567392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:40.567392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:40.567392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:40.567392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:40.567392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:40.567392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:40.567392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:40.567392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:40.567392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:40.567392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:40.569751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 180/343] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:40.570770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfbfbd-4aa5-4e31-8d66-631d724089c5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:40.614178 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d51bb-d5b4-4dfa-879e-302f35717847 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:40.649546 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6ac25-cbe0-448d-ae3b-07f628b554c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:40.650181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6ac25-cbe0-448d-ae3b-07f628b554c5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.650455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6ac25-cbe0-448d-ae3b-07f628b554c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.650723 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6ac25-cbe0-448d-ae3b-07f628b554c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.661561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6ac25-cbe0-448d-ae3b-07f628b554c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-25d6ac25-cbe0-448d-ae3b-07f628b554c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:40.666328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:40.667217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:40.668024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 164/344] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:40.669036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6ac25-cbe0-448d-ae3b-07f628b554c5 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:40.700383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:40.703193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:40.725110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfbfbd-4aa5-4e31-8d66-631d724089c5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:40.797601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6ac25-cbe0-448d-ae3b-07f628b554c5 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:40.833976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cd911-fa40-44c0-87e6-90eb6d833ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:40.834609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cd911-fa40-44c0-87e6-90eb6d833ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.834884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cd911-fa40-44c0-87e6-90eb6d833ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.835145 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1cd911-fa40-44c0-87e6-90eb6d833ed0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.838055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab32c18-cfab-4c4f-a055-b532bcc1817e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:40.838670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab32c18-cfab-4c4f-a055-b532bcc1817e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.839274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab32c18-cfab-4c4f-a055-b532bcc1817e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.839550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab32c18-cfab-4c4f-a055-b532bcc1817e None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.840319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab32c18-cfab-4c4f-a055-b532bcc1817e None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:40.840805 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab32c18-cfab-4c4f-a055-b532bcc1817e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:40.842284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab32c18-cfab-4c4f-a055-b532bcc1817e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:40.851407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 165/345] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:40.852423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab32c18-cfab-4c4f-a055-b532bcc1817e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:40.865417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1cd911-fa40-44c0-87e6-90eb6d833ed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-ec1cd911-fa40-44c0-87e6-90eb6d833ed0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:40.870187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:40.871065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 181/346] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:40.872383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cd911-fa40-44c0-87e6-90eb6d833ed0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:40.958701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:40.970831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab32c18-cfab-4c4f-a055-b532bcc1817e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.021829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36f13e-a470-4e36-8f01-a39b4a481d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.022428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36f13e-a470-4e36-8f01-a39b4a481d36 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.022829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36f13e-a470-4e36-8f01-a39b4a481d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.023081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36f13e-a470-4e36-8f01-a39b4a481d36 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.023669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36f13e-a470-4e36-8f01-a39b4a481d36 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:41.023969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36f13e-a470-4e36-8f01-a39b4a481d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:41.024917 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1cd911-fa40-44c0-87e6-90eb6d833ed0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.025429 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36f13e-a470-4e36-8f01-a39b4a481d36 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:41.037018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 166/347] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:41.038999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef14b2d-7110-4d3c-b2f5-f05ed79efbc2 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.039482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef14b2d-7110-4d3c-b2f5-f05ed79efbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.039967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef14b2d-7110-4d3c-b2f5-f05ed79efbc2 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.042470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 167/348] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:41.043640 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef14b2d-7110-4d3c-b2f5-f05ed79efbc2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.068569 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe2226-67df-4ccb-9a29-bbee7a691c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.069316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe2226-67df-4ccb-9a29-bbee7a691c19 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.069597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe2226-67df-4ccb-9a29-bbee7a691c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.070165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe2226-67df-4ccb-9a29-bbee7a691c19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.094666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.164168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe2226-67df-4ccb-9a29-bbee7a691c19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:41.166277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe2226-67df-4ccb-9a29-bbee7a691c19 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:41.186178 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef14b2d-7110-4d3c-b2f5-f05ed79efbc2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.234889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a5b78-4310-4da0-b233-1fa4d46f6410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.235875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a5b78-4310-4da0-b233-1fa4d46f6410 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.236344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a5b78-4310-4da0-b233-1fa4d46f6410 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.236841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a5b78-4310-4da0-b233-1fa4d46f6410 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.237700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a5b78-4310-4da0-b233-1fa4d46f6410 None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:41.238214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a5b78-4310-4da0-b233-1fa4d46f6410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:41.239794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a5b78-4310-4da0-b233-1fa4d46f6410 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:41.257374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 168/349] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:41.259576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a5b78-4310-4da0-b233-1fa4d46f6410 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.285185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.299180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 182/350] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:41.299890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe2226-67df-4ccb-9a29-bbee7a691c19 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.355210 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.395399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a5b78-4310-4da0-b233-1fa4d46f6410 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.414242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe2226-67df-4ccb-9a29-bbee7a691c19 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.448362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4f356-ad42-40d4-b636-63621142963d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.449295 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4f356-ad42-40d4-b636-63621142963d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.449736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4f356-ad42-40d4-b636-63621142963d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.450076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4f356-ad42-40d4-b636-63621142963d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.455977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584befac-41df-4293-8ba0-add4d667ed9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.456968 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584befac-41df-4293-8ba0-add4d667ed9c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.457464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584befac-41df-4293-8ba0-add4d667ed9c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.457949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584befac-41df-4293-8ba0-add4d667ed9c None admin] PATH_INFO: `/v3/users/2b3d4c7096134887a00e916ce00a0ee7` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.459392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584befac-41df-4293-8ba0-add4d667ed9c None admin] RBAC: Authorizing `identity:get_user(user_id=2b3d4c7096134887a00e916ce00a0ee7)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:41.461305 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584befac-41df-4293-8ba0-add4d667ed9c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:41.463362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4f356-ad42-40d4-b636-63621142963d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:41.463635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 183/351] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/users/2b3d4c7096134887a00e916ce00a0ee7 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:41.465496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d84c2-7f36-49f5-87a0-d8f83d10e5b5 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.465953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d84c2-7f36-49f5-87a0-d8f83d10e5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.466393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d84c2-7f36-49f5-87a0-d8f83d10e5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-52a4f356-ad42-40d4-b636-63621142963d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:41.469897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:41.470756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:41.471515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 169/352] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:41.472174 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4f356-ad42-40d4-b636-63621142963d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.488507 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-724d84c2-7f36-49f5-87a0-d8f83d10e5b5 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:41.489860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-724d84c2-7f36-49f5-87a0-d8f83d10e5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:41.571106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.589788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 184/353] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:41.590741 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d84c2-7f36-49f5-87a0-d8f83d10e5b5 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.608295 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4f356-ad42-40d4-b636-63621142963d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.644985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.654232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608c8cf-9bcc-49f1-b602-ea297b493aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.655538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608c8cf-9bcc-49f1-b602-ea297b493aba None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.656013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608c8cf-9bcc-49f1-b602-ea297b493aba None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.656479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608c8cf-9bcc-49f1-b602-ea297b493aba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.698845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d84c2-7f36-49f5-87a0-d8f83d10e5b5 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.736638 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efcc0f3-3c5e-4f7b-90d2-658fd29bb721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.737300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608c8cf-9bcc-49f1-b602-ea297b493aba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:41.737383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efcc0f3-3c5e-4f7b-90d2-658fd29bb721 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.737633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efcc0f3-3c5e-4f7b-90d2-658fd29bb721 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.737904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efcc0f3-3c5e-4f7b-90d2-658fd29bb721 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.738551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608c8cf-9bcc-49f1-b602-ea297b493aba None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:41.739024 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efcc0f3-3c5e-4f7b-90d2-658fd29bb721 None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:41.740472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efcc0f3-3c5e-4f7b-90d2-658fd29bb721 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:41.744637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 185/354] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:41.744637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efcc0f3-3c5e-4f7b-90d2-658fd29bb721 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.833512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.866712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 170/355] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:41.867758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608c8cf-9bcc-49f1-b602-ea297b493aba None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.887238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efcc0f3-3c5e-4f7b-90d2-658fd29bb721 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.907273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.927868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1ad32-c8aa-43ad-9921-d6d51908b9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.928474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1ad32-c8aa-43ad-9921-d6d51908b9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.928803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1ad32-c8aa-43ad-9921-d6d51908b9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.929074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1ad32-c8aa-43ad-9921-d6d51908b9e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.929801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1ad32-c8aa-43ad-9921-d6d51908b9e1 None admin] RBAC: Adding query filter params (role.id=26063cb085b6415989a19c8e7dc2c3c5, user.id=2b3d4c7096134887a00e916ce00a0ee7, scope.system=all) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:41.930103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1ad32-c8aa-43ad-9921-d6d51908b9e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:41.931562 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1ad32-c8aa-43ad-9921-d6d51908b9e1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:41.946344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 186/356] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/role_assignments?user.id=2b3d4c7096134887a00e916ce00a0ee7&scope.system=all&role.id=26063cb085b6415989a19c8e7dc2c3c5 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:41.947602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b509f7-c975-4e57-a0a5-aa789eb41e4b None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.947880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b509f7-c975-4e57-a0a5-aa789eb41e4b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.948826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b509f7-c975-4e57-a0a5-aa789eb41e4b None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.950919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 187/357] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:41.951964 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b509f7-c975-4e57-a0a5-aa789eb41e4b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.008108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608c8cf-9bcc-49f1-b602-ea297b493aba None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.060794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e2d22-567d-421c-83b5-d1d3db1b054e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.061597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e2d22-567d-421c-83b5-d1d3db1b054e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.061970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e2d22-567d-421c-83b5-d1d3db1b054e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.062339 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e2d22-567d-421c-83b5-d1d3db1b054e None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.063118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e2d22-567d-421c-83b5-d1d3db1b054e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:42.063515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e2d22-567d-421c-83b5-d1d3db1b054e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.065055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e2d22-567d-421c-83b5-d1d3db1b054e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.083541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 171/358] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:42.084580 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e2d22-567d-421c-83b5-d1d3db1b054e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.109760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b509f7-c975-4e57-a0a5-aa789eb41e4b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.139802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:42.157111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aebe55-92ad-41b1-9196-6bf02e312ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.158015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aebe55-92ad-41b1-9196-6bf02e312ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.158882 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aebe55-92ad-41b1-9196-6bf02e312ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.159295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aebe55-92ad-41b1-9196-6bf02e312ce3 None admin] PATH_INFO: `/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.166608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aebe55-92ad-41b1-9196-6bf02e312ce3 None admin] RBAC: Authorizing `identity:get_group(group_id=c06e2d3d5b06411c8cccc3da5822ce48)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.168286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aebe55-92ad-41b1-9196-6bf02e312ce3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.172028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 188/359] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:42.173065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aebe55-92ad-41b1-9196-6bf02e312ce3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.220285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e2d22-567d-421c-83b5-d1d3db1b054e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.224452 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:42.263369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6560e90-3006-4e9a-9826-3b59e6909126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.264005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6560e90-3006-4e9a-9826-3b59e6909126 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.264282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6560e90-3006-4e9a-9826-3b59e6909126 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.264549 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6560e90-3006-4e9a-9826-3b59e6909126 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.265728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6560e90-3006-4e9a-9826-3b59e6909126 None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.267894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6560e90-3006-4e9a-9826-3b59e6909126 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.269110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 172/360] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:42.270077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc2e0d-0507-4bf7-82e4-29711e77fc49 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.270357 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc2e0d-0507-4bf7-82e4-29711e77fc49 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.270662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc2e0d-0507-4bf7-82e4-29711e77fc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.289572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a3cc2e0d-0507-4bf7-82e4-29711e77fc49 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:42.290780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a3cc2e0d-0507-4bf7-82e4-29711e77fc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:42.302036 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aebe55-92ad-41b1-9196-6bf02e312ce3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.347980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dad89e-fe86-4fad-834b-0c0abda98cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.348608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dad89e-fe86-4fad-834b-0c0abda98cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.348923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dad89e-fe86-4fad-834b-0c0abda98cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.349199 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dad89e-fe86-4fad-834b-0c0abda98cf1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.361685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dad89e-fe86-4fad-834b-0c0abda98cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-70dad89e-fe86-4fad-834b-0c0abda98cf1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:42.366641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:42.367588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:42.368377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 189/361] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:42.371481 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 173/362] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:42.372252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dad89e-fe86-4fad-834b-0c0abda98cf1 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.372587 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc2e0d-0507-4bf7-82e4-29711e77fc49 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.423606 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:42.498722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:42.503876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dad89e-fe86-4fad-834b-0c0abda98cf1 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.509017 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc2e0d-0507-4bf7-82e4-29711e77fc49 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.554709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00fc2a-db50-4d7c-bbe3-dbd2ef61318c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.555334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00fc2a-db50-4d7c-bbe3-dbd2ef61318c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.555613 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00fc2a-db50-4d7c-bbe3-dbd2ef61318c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.555886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00fc2a-db50-4d7c-bbe3-dbd2ef61318c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.556639 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00fc2a-db50-4d7c-bbe3-dbd2ef61318c None admin] RBAC: Adding query filter params (group.id=c06e2d3d5b06411c8cccc3da5822ce48, role.id=3647def0715f4fd4991f82b06815ee38, scope.project.id=79572ca8d9504396b675929feb75e9f2) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:42.556975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00fc2a-db50-4d7c-bbe3-dbd2ef61318c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.558484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00fc2a-db50-4d7c-bbe3-dbd2ef61318c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.559666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fe940-2c2c-4c37-8a35-7c18c32fd386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.560275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fe940-2c2c-4c37-8a35-7c18c32fd386 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.560553 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fe940-2c2c-4c37-8a35-7c18c32fd386 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.560876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fe940-2c2c-4c37-8a35-7c18c32fd386 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.573326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 174/363] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/role_assignments?group.id=c06e2d3d5b06411c8cccc3da5822ce48&scope.project.id=79572ca8d9504396b675929feb75e9f2&role.id=3647def0715f4fd4991f82b06815ee38 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:42.573326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00fc2a-db50-4d7c-bbe3-dbd2ef61318c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.653346 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fe940-2c2c-4c37-8a35-7c18c32fd386 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.655168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fe940-2c2c-4c37-8a35-7c18c32fd386 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.708682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00fc2a-db50-4d7c-bbe3-dbd2ef61318c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.710528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:42.755322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e2169-9f74-49e9-9049-43a5120c09c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.755967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e2169-9f74-49e9-9049-43a5120c09c7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.756245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e2169-9f74-49e9-9049-43a5120c09c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.756522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e2169-9f74-49e9-9049-43a5120c09c7 None admin] PATH_INFO: `/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.759085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e2169-9f74-49e9-9049-43a5120c09c7 None admin] RBAC: Authorizing `identity:get_group(group_id=c06e2d3d5b06411c8cccc3da5822ce48)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.761288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e2169-9f74-49e9-9049-43a5120c09c7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.763925 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 175/364] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:42.764490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e2169-9f74-49e9-9049-43a5120c09c7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.772972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:42.779802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 190/365] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:42.781011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fe940-2c2c-4c37-8a35-7c18c32fd386 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.888670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e2169-9f74-49e9-9049-43a5120c09c7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.905253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fe940-2c2c-4c37-8a35-7c18c32fd386 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.929286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68309a5a-f670-4999-95dc-2c7f934706c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.931663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68309a5a-f670-4999-95dc-2c7f934706c3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.931663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68309a5a-f670-4999-95dc-2c7f934706c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.931663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68309a5a-f670-4999-95dc-2c7f934706c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.931663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68309a5a-f670-4999-95dc-2c7f934706c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:42.931663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68309a5a-f670-4999-95dc-2c7f934706c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.932795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68309a5a-f670-4999-95dc-2c7f934706c3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.947076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 176/366] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:42.947076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68309a5a-f670-4999-95dc-2c7f934706c3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.953414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcaf3f-5a25-44f2-a24a-b996638e1ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.954049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcaf3f-5a25-44f2-a24a-b996638e1ccc None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.954327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcaf3f-5a25-44f2-a24a-b996638e1ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.954972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcaf3f-5a25-44f2-a24a-b996638e1ccc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.967490 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcaf3f-5a25-44f2-a24a-b996638e1ccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-11fcaf3f-5a25-44f2-a24a-b996638e1ccc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:42.972281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:42.973170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:42.973939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 191/367] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:42.975603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcaf3f-5a25-44f2-a24a-b996638e1ccc None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.995991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.055295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.091543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68309a5a-f670-4999-95dc-2c7f934706c3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.104702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcaf3f-5a25-44f2-a24a-b996638e1ccc None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.135461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4632a-9a5d-4eaf-a063-f284639c2f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.136072 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4632a-9a5d-4eaf-a063-f284639c2f5c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.136350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4632a-9a5d-4eaf-a063-f284639c2f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.136620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4632a-9a5d-4eaf-a063-f284639c2f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.137414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4632a-9a5d-4eaf-a063-f284639c2f5c None admin] RBAC: Adding query filter params (group.id=c06e2d3d5b06411c8cccc3da5822ce48, role.id=3647def0715f4fd4991f82b06815ee38, scope.project.id=9549a5efb25e493d9a9b245fcc802166) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:43.137760 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4632a-9a5d-4eaf-a063-f284639c2f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.139300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4632a-9a5d-4eaf-a063-f284639c2f5c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.146015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 177/368] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/role_assignments?group.id=c06e2d3d5b06411c8cccc3da5822ce48&scope.project.id=9549a5efb25e493d9a9b245fcc802166&role.id=3647def0715f4fd4991f82b06815ee38 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:43.146724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4632a-9a5d-4eaf-a063-f284639c2f5c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.154638 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68195f71-c011-44bb-adb5-50abc4d6efe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.155408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68195f71-c011-44bb-adb5-50abc4d6efe2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.155765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68195f71-c011-44bb-adb5-50abc4d6efe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.156040 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68195f71-c011-44bb-adb5-50abc4d6efe2 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.157954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68195f71-c011-44bb-adb5-50abc4d6efe2 None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.159772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68195f71-c011-44bb-adb5-50abc4d6efe2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.162278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 192/369] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:43.162954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68195f71-c011-44bb-adb5-50abc4d6efe2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.278176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.287832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4632a-9a5d-4eaf-a063-f284639c2f5c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.314560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68195f71-c011-44bb-adb5-50abc4d6efe2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.344510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8988b3-e65d-4a99-af03-f47150d241af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.345492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8988b3-e65d-4a99-af03-f47150d241af None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.345964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8988b3-e65d-4a99-af03-f47150d241af None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.346400 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8988b3-e65d-4a99-af03-f47150d241af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.355529 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.366087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c14efa-36c9-499e-a2ba-a8358909c26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.366951 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c14efa-36c9-499e-a2ba-a8358909c26c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.367424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c14efa-36c9-499e-a2ba-a8358909c26c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.367860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c14efa-36c9-499e-a2ba-a8358909c26c None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.368735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c14efa-36c9-499e-a2ba-a8358909c26c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:43.369205 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c14efa-36c9-499e-a2ba-a8358909c26c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.370811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c14efa-36c9-499e-a2ba-a8358909c26c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.380247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 193/370] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:43.381409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c14efa-36c9-499e-a2ba-a8358909c26c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.439736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8988b3-e65d-4a99-af03-f47150d241af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.441623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8988b3-e65d-4a99-af03-f47150d241af None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.519242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c14efa-36c9-499e-a2ba-a8358909c26c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.567674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0514c-3bd7-4b85-8b76-80ff9d24a961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.568605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.570018 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0514c-3bd7-4b85-8b76-80ff9d24a961 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.570496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0514c-3bd7-4b85-8b76-80ff9d24a961 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.571252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0514c-3bd7-4b85-8b76-80ff9d24a961 None admin] PATH_INFO: `/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.573220 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0514c-3bd7-4b85-8b76-80ff9d24a961 None admin] RBAC: Authorizing `identity:get_group(group_id=c06e2d3d5b06411c8cccc3da5822ce48)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.575262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0514c-3bd7-4b85-8b76-80ff9d24a961 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.581878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 194/371] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:43.582659 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 178/372] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:43.583466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0514c-3bd7-4b85-8b76-80ff9d24a961 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.583570 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8988b3-e65d-4a99-af03-f47150d241af None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.662557 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.722993 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0514c-3bd7-4b85-8b76-80ff9d24a961 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.728726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8988b3-e65d-4a99-af03-f47150d241af None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.759988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9727de-ea1e-4fa9-ac71-06da55d45a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.760607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9727de-ea1e-4fa9-ac71-06da55d45a41 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.761097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9727de-ea1e-4fa9-ac71-06da55d45a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.761371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9727de-ea1e-4fa9-ac71-06da55d45a41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.778812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39791fe9-d693-4fd7-b4c5-fe610b436c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.779829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39791fe9-d693-4fd7-b4c5-fe610b436c02 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.780175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39791fe9-d693-4fd7-b4c5-fe610b436c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.780516 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39791fe9-d693-4fd7-b4c5-fe610b436c02 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.782510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39791fe9-d693-4fd7-b4c5-fe610b436c02 None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.784308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39791fe9-d693-4fd7-b4c5-fe610b436c02 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.787456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 179/373] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:43.788255 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39791fe9-d693-4fd7-b4c5-fe610b436c02 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.855696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9727de-ea1e-4fa9-ac71-06da55d45a41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.857601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9727de-ea1e-4fa9-ac71-06da55d45a41 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.881039 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.926633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39791fe9-d693-4fd7-b4c5-fe610b436c02 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.930285 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.977085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08147320-3540-4f2c-8cdf-679e5448742f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.978768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08147320-3540-4f2c-8cdf-679e5448742f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.979191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08147320-3540-4f2c-8cdf-679e5448742f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.979864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08147320-3540-4f2c-8cdf-679e5448742f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.983624 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08147320-3540-4f2c-8cdf-679e5448742f None admin] RBAC: Adding query filter params (group.id=c06e2d3d5b06411c8cccc3da5822ce48, role.id=16317a22e9c94a8886f02efb75ab456e, scope.project.id=79572ca8d9504396b675929feb75e9f2) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:43.983624 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08147320-3540-4f2c-8cdf-679e5448742f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.983624 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08147320-3540-4f2c-8cdf-679e5448742f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:44.001152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 180/374] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/role_assignments?group.id=c06e2d3d5b06411c8cccc3da5822ce48&scope.project.id=79572ca8d9504396b675929feb75e9f2&role.id=16317a22e9c94a8886f02efb75ab456e => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:44.001152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08147320-3540-4f2c-8cdf-679e5448742f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:44.005015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 195/375] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:44.011221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9727de-ea1e-4fa9-ac71-06da55d45a41 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:44.205235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08147320-3540-4f2c-8cdf-679e5448742f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:44.208102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9727de-ea1e-4fa9-ac71-06da55d45a41 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:44.250668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:44.261957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d335a6a-603b-45db-a00e-a565e7891a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:44.264100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d335a6a-603b-45db-a00e-a565e7891a50 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.265896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d335a6a-603b-45db-a00e-a565e7891a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.266713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424e6ac-1e89-427d-8fcb-fc9e196cb474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:44.267209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d335a6a-603b-45db-a00e-a565e7891a50 None admin] PATH_INFO: `/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.268067 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424e6ac-1e89-427d-8fcb-fc9e196cb474 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.269221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424e6ac-1e89-427d-8fcb-fc9e196cb474 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.270040 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d335a6a-603b-45db-a00e-a565e7891a50 None admin] RBAC: Authorizing `identity:get_group(group_id=c06e2d3d5b06411c8cccc3da5822ce48)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:44.270508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424e6ac-1e89-427d-8fcb-fc9e196cb474 None admin] PATH_INFO: `/v3/projects/66ac24841ce6483cbf8285ea30e957f8` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.272689 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d335a6a-603b-45db-a00e-a565e7891a50 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:44.273501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424e6ac-1e89-427d-8fcb-fc9e196cb474 None admin] RBAC: Authorizing `identity:get_project(project_id=66ac24841ce6483cbf8285ea30e957f8)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:44.275136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424e6ac-1e89-427d-8fcb-fc9e196cb474 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:44.277004 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 181/376] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48 => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:44.279992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 196/377] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:44 2022] GET /identity/v3/projects/66ac24841ce6483cbf8285ea30e957f8 => generated 344 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:44.285484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d335a6a-603b-45db-a00e-a565e7891a50 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:44.286984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424e6ac-1e89-427d-8fcb-fc9e196cb474 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:44.337653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:44.423227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424e6ac-1e89-427d-8fcb-fc9e196cb474 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:44.446105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d335a6a-603b-45db-a00e-a565e7891a50 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:44.466782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65897b06-b47e-432e-867b-c7cf8cffa916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:44.467357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65897b06-b47e-432e-867b-c7cf8cffa916 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.467681 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65897b06-b47e-432e-867b-c7cf8cffa916 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.468023 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65897b06-b47e-432e-867b-c7cf8cffa916 None admin] PATH_INFO: `/v3/groups/5ca37afbf4824d64883fd2a018489ff6` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.473430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65897b06-b47e-432e-867b-c7cf8cffa916 None admin] RBAC: Authorizing `identity:get_group(group_id=5ca37afbf4824d64883fd2a018489ff6)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:44.475234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65897b06-b47e-432e-867b-c7cf8cffa916 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:44.477580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 197/378] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:44 2022] GET /identity/v3/groups/5ca37afbf4824d64883fd2a018489ff6 => generated 229 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:44.483604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65897b06-b47e-432e-867b-c7cf8cffa916 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:44.494231 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd769c-e071-4173-8ab0-9bb3e49e4032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:44.494854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd769c-e071-4173-8ab0-9bb3e49e4032 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.495134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd769c-e071-4173-8ab0-9bb3e49e4032 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.495406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd769c-e071-4173-8ab0-9bb3e49e4032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.496146 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd769c-e071-4173-8ab0-9bb3e49e4032 None admin] RBAC: Adding query filter params (group.id=c06e2d3d5b06411c8cccc3da5822ce48, role.id=16317a22e9c94a8886f02efb75ab456e, scope.project.id=9549a5efb25e493d9a9b245fcc802166) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:44.496484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd769c-e071-4173-8ab0-9bb3e49e4032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:44.498104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd769c-e071-4173-8ab0-9bb3e49e4032 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:44.511025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 182/379] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:44 2022] GET /identity/v3/role_assignments?group.id=c06e2d3d5b06411c8cccc3da5822ce48&scope.project.id=9549a5efb25e493d9a9b245fcc802166&role.id=16317a22e9c94a8886f02efb75ab456e => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:44.615694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:44.630459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65897b06-b47e-432e-867b-c7cf8cffa916 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:44.641965 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bc1eb-a6b4-48db-9558-5839cccaeed2 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.642598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bc1eb-a6b4-48db-9558-5839cccaeed2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.643093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bc1eb-a6b4-48db-9558-5839cccaeed2 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.645959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 183/380] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:44.662589 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb7962-b668-40d6-8886-a4b38b739f81 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.662877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb7962-b668-40d6-8886-a4b38b739f81 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.663153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb7962-b668-40d6-8886-a4b38b739f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.682802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc901eab-8f53-4909-9d6a-95f72424c73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:44.683434 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc901eab-8f53-4909-9d6a-95f72424c73e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.683840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc901eab-8f53-4909-9d6a-95f72424c73e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.684305 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc901eab-8f53-4909-9d6a-95f72424c73e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.685606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc901eab-8f53-4909-9d6a-95f72424c73e None admin] RBAC: Adding query filter params (group.id=5ca37afbf4824d64883fd2a018489ff6, role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=66ac24841ce6483cbf8285ea30e957f8) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:44.685923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc901eab-8f53-4909-9d6a-95f72424c73e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:44.687424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc901eab-8f53-4909-9d6a-95f72424c73e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:44.698376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4beb7962-b668-40d6-8886-a4b38b739f81 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:44.699878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4beb7962-b668-40d6-8886-a4b38b739f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:44.704413 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 198/381] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:44 2022] GET /identity/v3/role_assignments?group.id=5ca37afbf4824d64883fd2a018489ff6&scope.project.id=66ac24841ce6483cbf8285ea30e957f8&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:44.783867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:44.802881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 184/382] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:45.226396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc901eab-8f53-4909-9d6a-95f72424c73e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:45.338277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc901eab-8f53-4909-9d6a-95f72424c73e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:45.374996 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e7c90-a7e7-4efc-b529-99bfb8226034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:45.375907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e7c90-a7e7-4efc-b529-99bfb8226034 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:45.376328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e7c90-a7e7-4efc-b529-99bfb8226034 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:45.376780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e7c90-a7e7-4efc-b529-99bfb8226034 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:45.389762 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7e7c90-a7e7-4efc-b529-99bfb8226034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-5a7e7c90-a7e7-4efc-b529-99bfb8226034 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:45.395537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:45.396684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:45.397485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:45.397485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:45.397485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:45.397485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:45.397485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:45.397485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:45.397485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:45.397485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:45.397485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:45.397485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:45.397485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:45.397485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:45.398080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51693316-a1a7-4ab0-ab73-0ed86d3a7b9e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:45.398080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51693316-a1a7-4ab0-ab73-0ed86d3a7b9e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:45.398080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51693316-a1a7-4ab0-ab73-0ed86d3a7b9e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:45.398269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 199/383] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:45.404872 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 185/384] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:45.405972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fab050-f86a-4e66-a58f-4d210bca0a06 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:45.406100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51693316-a1a7-4ab0-ab73-0ed86d3a7b9e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:45.406271 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fab050-f86a-4e66-a58f-4d210bca0a06 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:45.406522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fab050-f86a-4e66-a58f-4d210bca0a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:45.442850 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-65fab050-f86a-4e66-a58f-4d210bca0a06 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:45.444413 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-65fab050-f86a-4e66-a58f-4d210bca0a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:45.456972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:45.574435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51693316-a1a7-4ab0-ab73-0ed86d3a7b9e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:45.591720 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 200/385] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:45.612128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b5236-5186-4c27-b666-282e0c227301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:45.612960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b5236-5186-4c27-b666-282e0c227301 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:45.613416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b5236-5186-4c27-b666-282e0c227301 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:45.613835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b5236-5186-4c27-b666-282e0c227301 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:45.614642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b5236-5186-4c27-b666-282e0c227301 None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:45.615081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b5236-5186-4c27-b666-282e0c227301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:45.616612 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b5236-5186-4c27-b666-282e0c227301 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:45.625635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 186/386] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:45.631423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fab050-f86a-4e66-a58f-4d210bca0a06 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:45.735156 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fab050-f86a-4e66-a58f-4d210bca0a06 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:45.770917 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87de17f-b187-4124-8a55-1a90768f6cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:45.771859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87de17f-b187-4124-8a55-1a90768f6cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:45.772340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87de17f-b187-4124-8a55-1a90768f6cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:45.772812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87de17f-b187-4124-8a55-1a90768f6cd4 None admin] PATH_INFO: `/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:45.774145 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87de17f-b187-4124-8a55-1a90768f6cd4 None admin] RBAC: Authorizing `identity:get_group(group_id=c06e2d3d5b06411c8cccc3da5822ce48)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:45.775804 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87de17f-b187-4124-8a55-1a90768f6cd4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:45.777989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 201/387] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:45 2022] GET /identity/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:45.783720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b5236-5186-4c27-b666-282e0c227301 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:45.836882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:45.890906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b5236-5186-4c27-b666-282e0c227301 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:45.925983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96277331-d54f-4154-a0fa-3d058483018f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:45.926617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96277331-d54f-4154-a0fa-3d058483018f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:45.926897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96277331-d54f-4154-a0fa-3d058483018f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:45.927169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96277331-d54f-4154-a0fa-3d058483018f None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:45.928307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96277331-d54f-4154-a0fa-3d058483018f None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:45.929885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96277331-d54f-4154-a0fa-3d058483018f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:45.931966 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 187/388] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:45 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:45.937814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87de17f-b187-4124-8a55-1a90768f6cd4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:45.951411 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:46.045073 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87de17f-b187-4124-8a55-1a90768f6cd4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:46.079573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ddab9-b7b0-4389-a886-fce67423f471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:46.080237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ddab9-b7b0-4389-a886-fce67423f471 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:46.080516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ddab9-b7b0-4389-a886-fce67423f471 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:46.080815 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ddab9-b7b0-4389-a886-fce67423f471 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2/groups/c06e2d3d5b06411c8cccc3da5822ce48/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:46.082763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ddab9-b7b0-4389-a886-fce67423f471 None admin] RBAC: Authorizing `identity:create_grant(project_id=79572ca8d9504396b675929feb75e9f2, group_id=c06e2d3d5b06411c8cccc3da5822ce48, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:46.084214 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ddab9-b7b0-4389-a886-fce67423f471 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:46.086721 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96277331-d54f-4154-a0fa-3d058483018f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:46.124536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 202/389] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 24 12:19:45 2022] PUT /identity/v3/projects/79572ca8d9504396b675929feb75e9f2/groups/c06e2d3d5b06411c8cccc3da5822ce48/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:19:46.213694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96277331-d54f-4154-a0fa-3d058483018f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:46.234438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:46.254579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82c54e-6b43-4f56-983d-8d136a758f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:46.255220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82c54e-6b43-4f56-983d-8d136a758f97 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:46.255491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82c54e-6b43-4f56-983d-8d136a758f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:46.255757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82c54e-6b43-4f56-983d-8d136a758f97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:46.267243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82c54e-6b43-4f56-983d-8d136a758f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-5f82c54e-6b43-4f56-983d-8d136a758f97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:46.272727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:46.273635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:46.274407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 188/390] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:46.280405 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ddab9-b7b0-4389-a886-fce67423f471 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:46.303902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02293de5-a65f-4c0f-9330-2a66c8eaa602 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:46.304216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02293de5-a65f-4c0f-9330-2a66c8eaa602 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:46.304441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02293de5-a65f-4c0f-9330-2a66c8eaa602 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:46.306101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 189/391] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:46.313188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bff4d-b6c1-4f89-8f18-36add7690df3 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:46.313535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bff4d-b6c1-4f89-8f18-36add7690df3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:46.313894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bff4d-b6c1-4f89-8f18-36add7690df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:46.353635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-609bff4d-b6c1-4f89-8f18-36add7690df3 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:46.354919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-609bff4d-b6c1-4f89-8f18-36add7690df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:46.396284 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:46.435044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ddab9-b7b0-4389-a886-fce67423f471 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:46.442331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 190/392] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:46.474549 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c2f73-b018-4b12-836f-018ff3c44440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:46.475238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c2f73-b018-4b12-836f-018ff3c44440 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:46.475518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c2f73-b018-4b12-836f-018ff3c44440 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:46.475789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c2f73-b018-4b12-836f-018ff3c44440 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:46.476461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c2f73-b018-4b12-836f-018ff3c44440 None admin] RBAC: Adding query filter params (name=member) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:46.476800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c2f73-b018-4b12-836f-018ff3c44440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:46.478200 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c2f73-b018-4b12-836f-018ff3c44440 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:46.490270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 203/393] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:46.495557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609bff4d-b6c1-4f89-8f18-36add7690df3 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:46.649209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609bff4d-b6c1-4f89-8f18-36add7690df3 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:46.659651 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:46.686894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58fa59-89bf-4e50-ad54-ff561e1b05e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:46.687920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58fa59-89bf-4e50-ad54-ff561e1b05e2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:46.688419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58fa59-89bf-4e50-ad54-ff561e1b05e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:46.688925 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58fa59-89bf-4e50-ad54-ff561e1b05e2 None admin] PATH_INFO: `/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:46.690378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58fa59-89bf-4e50-ad54-ff561e1b05e2 None admin] RBAC: Authorizing `identity:get_group(group_id=c06e2d3d5b06411c8cccc3da5822ce48)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:46.692064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58fa59-89bf-4e50-ad54-ff561e1b05e2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:46.694256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 191/394] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:46 2022] GET /identity/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:46.701632 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c2f73-b018-4b12-836f-018ff3c44440 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:46.839203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58fa59-89bf-4e50-ad54-ff561e1b05e2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:46.846227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c2f73-b018-4b12-836f-018ff3c44440 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:46.877216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:46.885169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5656e7-cebf-46ff-97b3-19212fc8ce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:46.885811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5656e7-cebf-46ff-97b3-19212fc8ce1c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:46.886140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5656e7-cebf-46ff-97b3-19212fc8ce1c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:46.886404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5656e7-cebf-46ff-97b3-19212fc8ce1c None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:46.887538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5656e7-cebf-46ff-97b3-19212fc8ce1c None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:46.889668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5656e7-cebf-46ff-97b3-19212fc8ce1c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:46.892405 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 204/395] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:46 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:46.893736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cdc5b-05f6-4d9d-a045-2ea564912def None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:46.894019 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cdc5b-05f6-4d9d-a045-2ea564912def None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:46.894291 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cdc5b-05f6-4d9d-a045-2ea564912def None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:46.895530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 205/396] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:46.899838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cdc5b-05f6-4d9d-a045-2ea564912def None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:46.996681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58fa59-89bf-4e50-ad54-ff561e1b05e2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:47.017003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cdc5b-05f6-4d9d-a045-2ea564912def None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:47.035046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7afb7-3e30-42e6-bcfd-563b165e8bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:47.035046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7afb7-3e30-42e6-bcfd-563b165e8bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.035046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7afb7-3e30-42e6-bcfd-563b165e8bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.035046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7afb7-3e30-42e6-bcfd-563b165e8bc6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.055031 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7afb7-3e30-42e6-bcfd-563b165e8bc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:47.056018 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7c455-6f3c-44b2-9506-0aa716324b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:47.056704 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7c455-6f3c-44b2-9506-0aa716324b4f None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.056993 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7c455-6f3c-44b2-9506-0aa716324b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.057258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7c455-6f3c-44b2-9506-0aa716324b4f None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166/groups/c06e2d3d5b06411c8cccc3da5822ce48/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.059207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7c455-6f3c-44b2-9506-0aa716324b4f None admin] RBAC: Authorizing `identity:create_grant(project_id=9549a5efb25e493d9a9b245fcc802166, group_id=c06e2d3d5b06411c8cccc3da5822ce48, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-45d7afb7-3e30-42e6-bcfd-563b165e8bc6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:47.060517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:47.061480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:47.062321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:47.062321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:47.062321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:47.062321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:47.062321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:47.062321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:47.062321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:47.062321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:47.062321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:47.062321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:47.062321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:47.062321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:47.063259 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7c455-6f3c-44b2-9506-0aa716324b4f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:47.063392 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 192/397] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:47.064865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544b62d-6010-4315-85b2-eb8e7cedda2d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.065146 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544b62d-6010-4315-85b2-eb8e7cedda2d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.065547 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544b62d-6010-4315-85b2-eb8e7cedda2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.076707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 206/398] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 24 12:19:46 2022] PUT /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166/groups/c06e2d3d5b06411c8cccc3da5822ce48/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:19:47.078105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7c455-6f3c-44b2-9506-0aa716324b4f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:47.098067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6544b62d-6010-4315-85b2-eb8e7cedda2d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:47.099273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6544b62d-6010-4315-85b2-eb8e7cedda2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:47.150294 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:47.151333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:47.197195 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 193/399] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:47.198711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cc065-d5ea-4477-8e14-a69979c43615 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.198998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cc065-d5ea-4477-8e14-a69979c43615 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.199275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cc065-d5ea-4477-8e14-a69979c43615 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.200926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7c455-6f3c-44b2-9506-0aa716324b4f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:47.205133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 194/400] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:47.206709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baa10c-7cc4-4f2e-95ec-b733c2ced670 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.206997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baa10c-7cc4-4f2e-95ec-b733c2ced670 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.207274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baa10c-7cc4-4f2e-95ec-b733c2ced670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.240618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cef7f-d0a9-475c-9232-8e043a4375b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:47.241710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cef7f-d0a9-475c-9232-8e043a4375b0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.242188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cef7f-d0a9-475c-9232-8e043a4375b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.242669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cef7f-d0a9-475c-9232-8e043a4375b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.242837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-47baa10c-7cc4-4f2e-95ec-b733c2ced670 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:47.243712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cef7f-d0a9-475c-9232-8e043a4375b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:47.244071 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-47baa10c-7cc4-4f2e-95ec-b733c2ced670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:47.244429 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cef7f-d0a9-475c-9232-8e043a4375b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:47.246133 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cef7f-d0a9-475c-9232-8e043a4375b0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:47.257624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 207/401] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:47.265836 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cef7f-d0a9-475c-9232-8e043a4375b0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:47.366434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 195/402] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:47.396013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cef7f-d0a9-475c-9232-8e043a4375b0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:47.420847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:47.433216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85480d-2386-4f74-afa9-82d3e53a70a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:47.434419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85480d-2386-4f74-afa9-82d3e53a70a5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.435155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85480d-2386-4f74-afa9-82d3e53a70a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.435638 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85480d-2386-4f74-afa9-82d3e53a70a5 None admin] PATH_INFO: `/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.437060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85480d-2386-4f74-afa9-82d3e53a70a5 None admin] RBAC: Authorizing `identity:get_group(group_id=c06e2d3d5b06411c8cccc3da5822ce48)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:47.438806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85480d-2386-4f74-afa9-82d3e53a70a5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:47.440608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 208/403] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:47 2022] GET /identity/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:47.447882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baa10c-7cc4-4f2e-95ec-b733c2ced670 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:47.562409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85480d-2386-4f74-afa9-82d3e53a70a5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:47.613723 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:47.626426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baa10c-7cc4-4f2e-95ec-b733c2ced670 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:47.679924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ddcfc-7add-4ade-bcd0-d2c3aced82e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:47.680509 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ddcfc-7add-4ade-bcd0-d2c3aced82e0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.680828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ddcfc-7add-4ade-bcd0-d2c3aced82e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.681103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ddcfc-7add-4ade-bcd0-d2c3aced82e0 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.682228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ddcfc-7add-4ade-bcd0-d2c3aced82e0 None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:47.683781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ddcfc-7add-4ade-bcd0-d2c3aced82e0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:47.686007 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 196/404] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:47 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:47.694904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ddcfc-7add-4ade-bcd0-d2c3aced82e0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:47.768899 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85480d-2386-4f74-afa9-82d3e53a70a5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:47.813710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bbd77-2326-403e-bf8e-bda34d99d105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:47.814788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bbd77-2326-403e-bf8e-bda34d99d105 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.815070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bbd77-2326-403e-bf8e-bda34d99d105 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.815341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bbd77-2326-403e-bf8e-bda34d99d105 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.829255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059bbd77-2326-403e-bf8e-bda34d99d105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-059bbd77-2326-403e-bf8e-bda34d99d105 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:47.837386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:47.838229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:47.839601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 209/405] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:47.840795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ddcfc-7add-4ade-bcd0-d2c3aced82e0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:47.841198 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bbd77-2326-403e-bf8e-bda34d99d105 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:47.883020 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f28115-3341-471c-9d90-3b27d4ecc156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:47.883788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f28115-3341-471c-9d90-3b27d4ecc156 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.884121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f28115-3341-471c-9d90-3b27d4ecc156 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.884466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f28115-3341-471c-9d90-3b27d4ecc156 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2/groups/c06e2d3d5b06411c8cccc3da5822ce48/roles/16317a22e9c94a8886f02efb75ab456e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.887806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f28115-3341-471c-9d90-3b27d4ecc156 None admin] RBAC: Authorizing `identity:create_grant(project_id=79572ca8d9504396b675929feb75e9f2, group_id=c06e2d3d5b06411c8cccc3da5822ce48, role_id=16317a22e9c94a8886f02efb75ab456e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:47.889520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f28115-3341-471c-9d90-3b27d4ecc156 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:47.905771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:47.907900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 197/406] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 24 12:19:47 2022] PUT /identity/v3/projects/79572ca8d9504396b675929feb75e9f2/groups/c06e2d3d5b06411c8cccc3da5822ce48/roles/16317a22e9c94a8886f02efb75ab456e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:47.913664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f28115-3341-471c-9d90-3b27d4ecc156 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:47.966547 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:48.021198 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bbd77-2326-403e-bf8e-bda34d99d105 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.068730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04f278-3adc-47e4-822c-4bf0b0d7d9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.068730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04f278-3adc-47e4-822c-4bf0b0d7d9ad None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.068730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04f278-3adc-47e4-822c-4bf0b0d7d9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.068730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04f278-3adc-47e4-822c-4bf0b0d7d9ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.086043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f28115-3341-471c-9d90-3b27d4ecc156 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.087517 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04f278-3adc-47e4-822c-4bf0b0d7d9ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6f04f278-3adc-47e4-822c-4bf0b0d7d9ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:48.094439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:48.095646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:48.096548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:48.096548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:48.096548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:48.096548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:48.096548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:48.096548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:48.096548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:48.096548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:48.096548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:48.096548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:48.096548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:48.096548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:48.105169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 210/407] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:48.117743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04f278-3adc-47e4-822c-4bf0b0d7d9ad None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.142058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eca36-7c4a-4981-a405-6b74c51236f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.142058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eca36-7c4a-4981-a405-6b74c51236f8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.142058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eca36-7c4a-4981-a405-6b74c51236f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.142459 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5eca36-7c4a-4981-a405-6b74c51236f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.142833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5eca36-7c4a-4981-a405-6b74c51236f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:48.143135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5eca36-7c4a-4981-a405-6b74c51236f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.144641 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5eca36-7c4a-4981-a405-6b74c51236f8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.156399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 198/408] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:48.161875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eca36-7c4a-4981-a405-6b74c51236f8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.261260 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04f278-3adc-47e4-822c-4bf0b0d7d9ad None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.264062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:48.273626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:48.282098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5eca36-7c4a-4981-a405-6b74c51236f8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.300546 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0945943-b19b-483d-a4ac-73c23cae8998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.301321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0945943-b19b-483d-a4ac-73c23cae8998 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.301602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0945943-b19b-483d-a4ac-73c23cae8998 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.301868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0945943-b19b-483d-a4ac-73c23cae8998 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.302629 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0945943-b19b-483d-a4ac-73c23cae8998 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:48.302930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0945943-b19b-483d-a4ac-73c23cae8998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.304347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0945943-b19b-483d-a4ac-73c23cae8998 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.313719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 211/409] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:48.321648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0945943-b19b-483d-a4ac-73c23cae8998 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.341199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0d867-4bcd-4a31-94f0-c8f156adedc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.342089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0d867-4bcd-4a31-94f0-c8f156adedc9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.342671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0d867-4bcd-4a31-94f0-c8f156adedc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.343084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0d867-4bcd-4a31-94f0-c8f156adedc9 None admin] PATH_INFO: `/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.344378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0d867-4bcd-4a31-94f0-c8f156adedc9 None admin] RBAC: Authorizing `identity:get_group(group_id=c06e2d3d5b06411c8cccc3da5822ce48)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.346026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0d867-4bcd-4a31-94f0-c8f156adedc9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.349874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 199/410] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:48 2022] GET /identity/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:48.354381 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0d867-4bcd-4a31-94f0-c8f156adedc9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.487376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0945943-b19b-483d-a4ac-73c23cae8998 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.529002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5ca0c-b534-4dbd-86ef-ab86838a8f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.531335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5ca0c-b534-4dbd-86ef-ab86838a8f3b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.532448 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5ca0c-b534-4dbd-86ef-ab86838a8f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.533092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5ca0c-b534-4dbd-86ef-ab86838a8f3b None admin] PATH_INFO: `/v3/groups/5ca37afbf4824d64883fd2a018489ff6` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.536821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5ca0c-b534-4dbd-86ef-ab86838a8f3b None admin] RBAC: Authorizing `identity:get_group(group_id=5ca37afbf4824d64883fd2a018489ff6)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.539223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5ca0c-b534-4dbd-86ef-ab86838a8f3b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.551783 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 212/411] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:48 2022] GET /identity/v3/groups/5ca37afbf4824d64883fd2a018489ff6 => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:48.554389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5ca0c-b534-4dbd-86ef-ab86838a8f3b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.581779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0d867-4bcd-4a31-94f0-c8f156adedc9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.614257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:48.631663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a3b7f-6f69-4c20-8668-4c1edeb8146e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.632395 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a3b7f-6f69-4c20-8668-4c1edeb8146e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.632840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a3b7f-6f69-4c20-8668-4c1edeb8146e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.633118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a3b7f-6f69-4c20-8668-4c1edeb8146e None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.634236 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a3b7f-6f69-4c20-8668-4c1edeb8146e None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.635706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a3b7f-6f69-4c20-8668-4c1edeb8146e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.638048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 200/412] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:48 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:48.639233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7709bdb-d65c-4e4d-a553-fc49312316a5 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.639514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7709bdb-d65c-4e4d-a553-fc49312316a5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.639787 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7709bdb-d65c-4e4d-a553-fc49312316a5 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.646009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 201/413] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:48.646931 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7709bdb-d65c-4e4d-a553-fc49312316a5 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.672230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5ca0c-b534-4dbd-86ef-ab86838a8f3b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.675285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:48.707312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3121e4d-8d05-47d5-98d3-48b39e171f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.707941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3121e4d-8d05-47d5-98d3-48b39e171f46 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.708216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3121e4d-8d05-47d5-98d3-48b39e171f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.708483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3121e4d-8d05-47d5-98d3-48b39e171f46 None admin] PATH_INFO: `/v3/projects/66ac24841ce6483cbf8285ea30e957f8` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.709649 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3121e4d-8d05-47d5-98d3-48b39e171f46 None admin] RBAC: Authorizing `identity:get_project(project_id=66ac24841ce6483cbf8285ea30e957f8)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.711091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3121e4d-8d05-47d5-98d3-48b39e171f46 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.713046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 213/414] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:48 2022] GET /identity/v3/projects/66ac24841ce6483cbf8285ea30e957f8 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:48.714505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0dc3e2-b230-4cf9-8d76-36cb44fe17ee None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.714780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0dc3e2-b230-4cf9-8d76-36cb44fe17ee None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.715081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0dc3e2-b230-4cf9-8d76-36cb44fe17ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.733596 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1e0dc3e2-b230-4cf9-8d76-36cb44fe17ee None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:48.734791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1e0dc3e2-b230-4cf9-8d76-36cb44fe17ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:48.764993 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7709bdb-d65c-4e4d-a553-fc49312316a5 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.803781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 214/415] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:48.804035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93caded4-d81d-4740-b10e-0b01c91fd427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.804508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93caded4-d81d-4740-b10e-0b01c91fd427 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.804684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0dc3e2-b230-4cf9-8d76-36cb44fe17ee None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.804830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93caded4-d81d-4740-b10e-0b01c91fd427 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.805128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93caded4-d81d-4740-b10e-0b01c91fd427 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166/groups/c06e2d3d5b06411c8cccc3da5822ce48/roles/16317a22e9c94a8886f02efb75ab456e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.807635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93caded4-d81d-4740-b10e-0b01c91fd427 None admin] RBAC: Authorizing `identity:create_grant(project_id=9549a5efb25e493d9a9b245fcc802166, group_id=c06e2d3d5b06411c8cccc3da5822ce48, role_id=16317a22e9c94a8886f02efb75ab456e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.809602 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93caded4-d81d-4740-b10e-0b01c91fd427 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.835203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 202/416] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 24 12:19:48 2022] PUT /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166/groups/c06e2d3d5b06411c8cccc3da5822ce48/roles/16317a22e9c94a8886f02efb75ab456e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:48.847299 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:48.913898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0dc3e2-b230-4cf9-8d76-36cb44fe17ee None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.952387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65261971-b9f8-4149-95c2-ade95b198438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.953162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65261971-b9f8-4149-95c2-ade95b198438 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.953441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65261971-b9f8-4149-95c2-ade95b198438 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.953712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65261971-b9f8-4149-95c2-ade95b198438 None admin] PATH_INFO: `/v3/projects/66ac24841ce6483cbf8285ea30e957f8/groups/5ca37afbf4824d64883fd2a018489ff6/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.955898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65261971-b9f8-4149-95c2-ade95b198438 None admin] RBAC: Authorizing `identity:create_grant(project_id=66ac24841ce6483cbf8285ea30e957f8, group_id=5ca37afbf4824d64883fd2a018489ff6, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.957487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65261971-b9f8-4149-95c2-ade95b198438 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.975306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 215/417] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 24 12:19:48 2022] PUT /identity/v3/projects/66ac24841ce6483cbf8285ea30e957f8/groups/5ca37afbf4824d64883fd2a018489ff6/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:49.158785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93caded4-d81d-4740-b10e-0b01c91fd427 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:49.219399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10a31-6096-4942-9c21-ba7ee06114f3 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.219678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10a31-6096-4942-9c21-ba7ee06114f3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.219956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a10a31-6096-4942-9c21-ba7ee06114f3 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.223894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 216/418] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:49.226760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3d4fc-32a5-4313-ad3c-5f3e097a329f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.227048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3d4fc-32a5-4313-ad3c-5f3e097a329f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.227324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3d4fc-32a5-4313-ad3c-5f3e097a329f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.249682 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-98a3d4fc-32a5-4313-ad3c-5f3e097a329f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:49.251027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-98a3d4fc-32a5-4313-ad3c-5f3e097a329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:49.258391 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:49.281425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93caded4-d81d-4740-b10e-0b01c91fd427 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:49.324779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f77634-633d-4468-b330-412b1b1a6405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:49.325937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f77634-633d-4468-b330-412b1b1a6405 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.326372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f77634-633d-4468-b330-412b1b1a6405 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.326845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f77634-633d-4468-b330-412b1b1a6405 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.345729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f77634-633d-4468-b330-412b1b1a6405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b6f77634-633d-4468-b330-412b1b1a6405 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:49.351387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:49.352506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:49.353527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:49.353527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:49.353527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:49.353527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:49.353527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:49.353527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:49.353527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:49.353527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:49.353527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:49.353527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:49.353527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:49.353527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:49.356081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 203/419] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:49.362407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f77634-633d-4468-b330-412b1b1a6405 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:49.363907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 217/420] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:49.493172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f77634-633d-4468-b330-412b1b1a6405 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:49.544386 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39e216-57b4-46fe-a4f7-e587d046ccb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:49.545028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39e216-57b4-46fe-a4f7-e587d046ccb1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.545310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39e216-57b4-46fe-a4f7-e587d046ccb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.545582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39e216-57b4-46fe-a4f7-e587d046ccb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.546270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39e216-57b4-46fe-a4f7-e587d046ccb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:49.546576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39e216-57b4-46fe-a4f7-e587d046ccb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:49.547968 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39e216-57b4-46fe-a4f7-e587d046ccb1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:49.560693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 204/421] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:49.566042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3d4fc-32a5-4313-ad3c-5f3e097a329f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:49.613823 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:49.710134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3d4fc-32a5-4313-ad3c-5f3e097a329f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:49.757741 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd368b-c298-42a4-b6e8-c5d27f2841ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:49.758699 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd368b-c298-42a4-b6e8-c5d27f2841ee None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.759153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd368b-c298-42a4-b6e8-c5d27f2841ee None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.759555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39e216-57b4-46fe-a4f7-e587d046ccb1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:49.759694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd368b-c298-42a4-b6e8-c5d27f2841ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.763700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:49.909704 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd368b-c298-42a4-b6e8-c5d27f2841ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:49.911188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd368b-c298-42a4-b6e8-c5d27f2841ee None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:49.913774 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39e216-57b4-46fe-a4f7-e587d046ccb1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:49.952908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1b2c6-76a4-41f2-9c6a-024968880288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:49.953540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1b2c6-76a4-41f2-9c6a-024968880288 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.953816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1b2c6-76a4-41f2-9c6a-024968880288 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.954266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1b2c6-76a4-41f2-9c6a-024968880288 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.966361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1b2c6-76a4-41f2-9c6a-024968880288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-04e1b2c6-76a4-41f2-9c6a-024968880288 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:49.972358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:49.973480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:49.974424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 205/422] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:49.984405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1b2c6-76a4-41f2-9c6a-024968880288 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:49.994051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:50.064329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:50.098193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1b2c6-76a4-41f2-9c6a-024968880288 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:50.135171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f57034-6ddf-4062-9356-e3e1f4e49c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:50.135872 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f57034-6ddf-4062-9356-e3e1f4e49c9f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:50.136221 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f57034-6ddf-4062-9356-e3e1f4e49c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:50.136564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f57034-6ddf-4062-9356-e3e1f4e49c9f None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:50.137536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f57034-6ddf-4062-9356-e3e1f4e49c9f None admin] RBAC: Adding query filter params (name=member) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:50.138167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f57034-6ddf-4062-9356-e3e1f4e49c9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:50.139707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f57034-6ddf-4062-9356-e3e1f4e49c9f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:50.148725 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 206/423] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:50.154054 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f57034-6ddf-4062-9356-e3e1f4e49c9f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:50.213658 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 218/424] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:50.220311 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd368b-c298-42a4-b6e8-c5d27f2841ee None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:50.269948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f57034-6ddf-4062-9356-e3e1f4e49c9f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:50.305041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:50.321466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e06355-c6e1-42eb-a67f-19388d97dc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:50.326468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e06355-c6e1-42eb-a67f-19388d97dc24 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:50.330482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e06355-c6e1-42eb-a67f-19388d97dc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:50.336395 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e06355-c6e1-42eb-a67f-19388d97dc24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:50.358773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd368b-c298-42a4-b6e8-c5d27f2841ee None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:50.391001 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:50.398915 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d6c3e-b771-458c-b966-6ebff36f2b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:50.399613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d6c3e-b771-458c-b966-6ebff36f2b26 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:50.399892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d6c3e-b771-458c-b966-6ebff36f2b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:50.400161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d6c3e-b771-458c-b966-6ebff36f2b26 None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:50.401517 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d6c3e-b771-458c-b966-6ebff36f2b26 None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:50.403109 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d6c3e-b771-458c-b966-6ebff36f2b26 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:50.406342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 219/425] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:50 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:50.408150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf20f2-97ee-4e5a-bbcc-d8f94582d96c None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:50.408437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf20f2-97ee-4e5a-bbcc-d8f94582d96c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:50.409028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf20f2-97ee-4e5a-bbcc-d8f94582d96c None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:50.410578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 220/426] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:50.412573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf20f2-97ee-4e5a-bbcc-d8f94582d96c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:50.443033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e06355-c6e1-42eb-a67f-19388d97dc24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:50.444997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e06355-c6e1-42eb-a67f-19388d97dc24 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:50.533919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf20f2-97ee-4e5a-bbcc-d8f94582d96c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:50.577217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e179b-dba2-4fb3-aa69-6d904f17d411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:50.578149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e179b-dba2-4fb3-aa69-6d904f17d411 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:50.578604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e179b-dba2-4fb3-aa69-6d904f17d411 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:50.579047 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e179b-dba2-4fb3-aa69-6d904f17d411 None admin] PATH_INFO: `/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:50.579532 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 207/427] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:50.580877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09bde8-1d27-4447-84be-cab7f5d884f4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:50.580995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e179b-dba2-4fb3-aa69-6d904f17d411 None admin] RBAC: Authorizing `identity:get_group(group_id=c06e2d3d5b06411c8cccc3da5822ce48)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:50.581156 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09bde8-1d27-4447-84be-cab7f5d884f4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:50.581432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09bde8-1d27-4447-84be-cab7f5d884f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:50.583842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e179b-dba2-4fb3-aa69-6d904f17d411 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:50.587217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 221/428] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:50 2022] GET /identity/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:50.590676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e179b-dba2-4fb3-aa69-6d904f17d411 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:50.607517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9e09bde8-1d27-4447-84be-cab7f5d884f4 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:50.609188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9e09bde8-1d27-4447-84be-cab7f5d884f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:50.645013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:50.663975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:50.685032 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 208/429] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:50.686443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09bde8-1d27-4447-84be-cab7f5d884f4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:50.707361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e179b-dba2-4fb3-aa69-6d904f17d411 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:50.758586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c80620-bde1-4dfc-9f94-62e8f6c98e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:50.760805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c80620-bde1-4dfc-9f94-62e8f6c98e4d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:50.762484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c80620-bde1-4dfc-9f94-62e8f6c98e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:50.763107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c80620-bde1-4dfc-9f94-62e8f6c98e4d None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:50.766435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c80620-bde1-4dfc-9f94-62e8f6c98e4d None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:50.768136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c80620-bde1-4dfc-9f94-62e8f6c98e4d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:50.771396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 222/430] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:50 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:50.777097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c80620-bde1-4dfc-9f94-62e8f6c98e4d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:50.831871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09bde8-1d27-4447-84be-cab7f5d884f4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:50.890265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e8d54-3485-4e00-88f4-8f660e455deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:50.891264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e8d54-3485-4e00-88f4-8f660e455deb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:50.891745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e8d54-3485-4e00-88f4-8f660e455deb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:50.892206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e8d54-3485-4e00-88f4-8f660e455deb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:50.893166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e8d54-3485-4e00-88f4-8f660e455deb None admin] RBAC: Adding query filter params (group.id=c06e2d3d5b06411c8cccc3da5822ce48, role.id=3647def0715f4fd4991f82b06815ee38, scope.project.id=79572ca8d9504396b675929feb75e9f2) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:50.893657 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e8d54-3485-4e00-88f4-8f660e455deb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:50.904682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e8d54-3485-4e00-88f4-8f660e455deb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:50.911646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 209/431] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:50 2022] GET /identity/v3/role_assignments?group.id=c06e2d3d5b06411c8cccc3da5822ce48&scope.project.id=79572ca8d9504396b675929feb75e9f2&role.id=3647def0715f4fd4991f82b06815ee38 => generated 611 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:50.915936 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c80620-bde1-4dfc-9f94-62e8f6c98e4d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:50.950045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:50.963186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561f074-ee23-48c9-991f-f12cfc60031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:50.964151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561f074-ee23-48c9-991f-f12cfc60031f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:50.964432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561f074-ee23-48c9-991f-f12cfc60031f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:50.964733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561f074-ee23-48c9-991f-f12cfc60031f None admin] PATH_INFO: `/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:50.969509 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561f074-ee23-48c9-991f-f12cfc60031f None admin] RBAC: Authorizing `identity:get_group(group_id=c06e2d3d5b06411c8cccc3da5822ce48)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:50.971002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561f074-ee23-48c9-991f-f12cfc60031f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:50.979576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e8d54-3485-4e00-88f4-8f660e455deb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:50.983379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 223/432] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:50 2022] GET /identity/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:51.028678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:51.114640 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e8d54-3485-4e00-88f4-8f660e455deb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:51.123854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561f074-ee23-48c9-991f-f12cfc60031f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:51.151867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36367382-33d4-42cb-bdf4-dd7ab292059f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:51.152936 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36367382-33d4-42cb-bdf4-dd7ab292059f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.153671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36367382-33d4-42cb-bdf4-dd7ab292059f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.154150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36367382-33d4-42cb-bdf4-dd7ab292059f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.155079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36367382-33d4-42cb-bdf4-dd7ab292059f None admin] RBAC: Adding query filter params (group.id=c06e2d3d5b06411c8cccc3da5822ce48, role.id=3647def0715f4fd4991f82b06815ee38, scope.project.id=9549a5efb25e493d9a9b245fcc802166) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:51.155569 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36367382-33d4-42cb-bdf4-dd7ab292059f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:51.157263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36367382-33d4-42cb-bdf4-dd7ab292059f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:51.164297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 210/433] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:50 2022] GET /identity/v3/role_assignments?group.id=c06e2d3d5b06411c8cccc3da5822ce48&scope.project.id=9549a5efb25e493d9a9b245fcc802166&role.id=3647def0715f4fd4991f82b06815ee38 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:51.251712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561f074-ee23-48c9-991f-f12cfc60031f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:51.290804 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4239751-5c32-4219-bb09-8dd2121ba5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:51.291449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4239751-5c32-4219-bb09-8dd2121ba5af None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.291688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aedaae-7098-47a0-a6df-17490e708a0b None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.292145 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4239751-5c32-4219-bb09-8dd2121ba5af None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.292223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aedaae-7098-47a0-a6df-17490e708a0b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.292289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4239751-5c32-4219-bb09-8dd2121ba5af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.292393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aedaae-7098-47a0-a6df-17490e708a0b None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.294321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 211/434] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:51.300243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12978bb-8acd-422d-a441-f617e9c3ce8f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.300889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12978bb-8acd-422d-a441-f617e9c3ce8f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.301368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12978bb-8acd-422d-a441-f617e9c3ce8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.308617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4239751-5c32-4219-bb09-8dd2121ba5af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-e4239751-5c32-4219-bb09-8dd2121ba5af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:51.314365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:51.315310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:51.316081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 224/435] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:51.319317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9429f0-bbe1-418d-ae7c-afebf5e1fd83 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.319317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9429f0-bbe1-418d-ae7c-afebf5e1fd83 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.319317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9429f0-bbe1-418d-ae7c-afebf5e1fd83 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.323214 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 225/436] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:51.324612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9429f0-bbe1-418d-ae7c-afebf5e1fd83 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:51.324816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e12978bb-8acd-422d-a441-f617e9c3ce8f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:51.327661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e12978bb-8acd-422d-a441-f617e9c3ce8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:51.411503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:51.430206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 212/437] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:51.431473 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f1e8-4087-47c2-ada9-f29a609f0403 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.431755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f1e8-4087-47c2-ada9-f29a609f0403 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.432027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f1e8-4087-47c2-ada9-f29a609f0403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.461306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9429f0-bbe1-418d-ae7c-afebf5e1fd83 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:51.465884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c691f1e8-4087-47c2-ada9-f29a609f0403 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:51.467056 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c691f1e8-4087-47c2-ada9-f29a609f0403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:51.481017 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:51.509298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652584d-d71e-41cb-ac90-ab0a616c0aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:51.509895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652584d-d71e-41cb-ac90-ab0a616c0aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.510166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652584d-d71e-41cb-ac90-ab0a616c0aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.510428 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652584d-d71e-41cb-ac90-ab0a616c0aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.511090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652584d-d71e-41cb-ac90-ab0a616c0aa3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:51.511404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652584d-d71e-41cb-ac90-ab0a616c0aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:51.513009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652584d-d71e-41cb-ac90-ab0a616c0aa3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:51.522275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 226/438] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:51.529103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652584d-d71e-41cb-ac90-ab0a616c0aa3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:51.558131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 213/439] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:51.651193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652584d-d71e-41cb-ac90-ab0a616c0aa3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:51.675677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:51.689181 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234219be-3f1a-4e77-8413-810ff9ae577e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:51.689781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234219be-3f1a-4e77-8413-810ff9ae577e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.690054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234219be-3f1a-4e77-8413-810ff9ae577e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.690354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234219be-3f1a-4e77-8413-810ff9ae577e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.703185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691f1e8-4087-47c2-ada9-f29a609f0403 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:51.763449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234219be-3f1a-4e77-8413-810ff9ae577e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:51.765051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234219be-3f1a-4e77-8413-810ff9ae577e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:51.808160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691f1e8-4087-47c2-ada9-f29a609f0403 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:51.843428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e06032-25a1-4270-a778-f6c27bf7f130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:51.844061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e06032-25a1-4270-a778-f6c27bf7f130 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.844341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e06032-25a1-4270-a778-f6c27bf7f130 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.844612 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e06032-25a1-4270-a778-f6c27bf7f130 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.854623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e06032-25a1-4270-a778-f6c27bf7f130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-13e06032-25a1-4270-a778-f6c27bf7f130 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:51.859436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:51.860452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 214/440] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:51.861316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e06032-25a1-4270-a778-f6c27bf7f130 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:51.865587 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:51.871165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 227/441] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:51.872155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234219be-3f1a-4e77-8413-810ff9ae577e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:51.895411 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:51.964358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e06032-25a1-4270-a778-f6c27bf7f130 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:51.969872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234219be-3f1a-4e77-8413-810ff9ae577e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:51.998858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcffe1b-a20c-4d29-b19f-e2589fb2f270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:51.999498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcffe1b-a20c-4d29-b19f-e2589fb2f270 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.000059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcffe1b-a20c-4d29-b19f-e2589fb2f270 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.000334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcffe1b-a20c-4d29-b19f-e2589fb2f270 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.004570 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2776d2-7297-48fe-9562-7f76711ed470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.005200 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2776d2-7297-48fe-9562-7f76711ed470 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.005476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2776d2-7297-48fe-9562-7f76711ed470 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.005796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2776d2-7297-48fe-9562-7f76711ed470 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.006404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2776d2-7297-48fe-9562-7f76711ed470 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:52.006702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2776d2-7297-48fe-9562-7f76711ed470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.008068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2776d2-7297-48fe-9562-7f76711ed470 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.010834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcffe1b-a20c-4d29-b19f-e2589fb2f270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-6bcffe1b-a20c-4d29-b19f-e2589fb2f270 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:52.015563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:52.016466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:52.017579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:52.017579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:52.017579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:52.017579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:52.017579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:52.017579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:52.017579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:52.017579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:52.017579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:52.017579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:52.017579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:52.017579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:52.018333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 228/442] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:52.018416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 215/443] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:52.018494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2776d2-7297-48fe-9562-7f76711ed470 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.021967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcffe1b-a20c-4d29-b19f-e2589fb2f270 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.085590 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.101011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.116872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2776d2-7297-48fe-9562-7f76711ed470 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.125217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcffe1b-a20c-4d29-b19f-e2589fb2f270 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.156559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ed09f-7b5d-40d3-aa98-060bbb85c30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.157321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ed09f-7b5d-40d3-aa98-060bbb85c30a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.157662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ed09f-7b5d-40d3-aa98-060bbb85c30a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.157947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ed09f-7b5d-40d3-aa98-060bbb85c30a None admin] PATH_INFO: `/v3/projects/79572ca8d9504396b675929feb75e9f2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.159143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ed09f-7b5d-40d3-aa98-060bbb85c30a None admin] RBAC: Authorizing `identity:get_project(project_id=79572ca8d9504396b675929feb75e9f2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.160894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ed09f-7b5d-40d3-aa98-060bbb85c30a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.162583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 229/444] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/projects/79572ca8d9504396b675929feb75e9f2 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:52.163328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ed09f-7b5d-40d3-aa98-060bbb85c30a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.164830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee713b-8e7c-4597-a02b-721adfde5a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.165428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee713b-8e7c-4597-a02b-721adfde5a59 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.165722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee713b-8e7c-4597-a02b-721adfde5a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.166128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee713b-8e7c-4597-a02b-721adfde5a59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.238486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee713b-8e7c-4597-a02b-721adfde5a59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.239872 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee713b-8e7c-4597-a02b-721adfde5a59 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.265511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ed09f-7b5d-40d3-aa98-060bbb85c30a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.298749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2796117-c570-42de-ad0d-807a9ba56e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.299350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2796117-c570-42de-ad0d-807a9ba56e10 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.299635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2796117-c570-42de-ad0d-807a9ba56e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.299786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.300216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2796117-c570-42de-ad0d-807a9ba56e10 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.300625 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2796117-c570-42de-ad0d-807a9ba56e10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:52.300945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2796117-c570-42de-ad0d-807a9ba56e10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.302432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2796117-c570-42de-ad0d-807a9ba56e10 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.311043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 230/445] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:52.311838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2796117-c570-42de-ad0d-807a9ba56e10 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.327538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.344094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 216/446] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:19:52.344989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee713b-8e7c-4597-a02b-721adfde5a59 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.416928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2796117-c570-42de-ad0d-807a9ba56e10 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.444177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee713b-8e7c-4597-a02b-721adfde5a59 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.461771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da5424-c412-4fdc-a5a1-66eaa9e56bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.462401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da5424-c412-4fdc-a5a1-66eaa9e56bdf None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.462682 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da5424-c412-4fdc-a5a1-66eaa9e56bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.462955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da5424-c412-4fdc-a5a1-66eaa9e56bdf None admin] PATH_INFO: `/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.464399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da5424-c412-4fdc-a5a1-66eaa9e56bdf None admin] RBAC: Authorizing `identity:get_group(group_id=c06e2d3d5b06411c8cccc3da5822ce48)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.465929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da5424-c412-4fdc-a5a1-66eaa9e56bdf None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.467510 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 231/447] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:52.468270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da5424-c412-4fdc-a5a1-66eaa9e56bdf None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.488356 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01485d6-6db6-45c1-a09a-0cc3251e5ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.489049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01485d6-6db6-45c1-a09a-0cc3251e5ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.489272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01485d6-6db6-45c1-a09a-0cc3251e5ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.489539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01485d6-6db6-45c1-a09a-0cc3251e5ee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.522331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.556093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.561521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01485d6-6db6-45c1-a09a-0cc3251e5ee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.563375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01485d6-6db6-45c1-a09a-0cc3251e5ee7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.575653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da5424-c412-4fdc-a5a1-66eaa9e56bdf None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.609486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8208d5b-4c31-4a5e-a6fd-bc36f3130832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.610111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8208d5b-4c31-4a5e-a6fd-bc36f3130832 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.610388 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8208d5b-4c31-4a5e-a6fd-bc36f3130832 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.610654 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8208d5b-4c31-4a5e-a6fd-bc36f3130832 None admin] PATH_INFO: `/v3/projects/9549a5efb25e493d9a9b245fcc802166` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.611758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8208d5b-4c31-4a5e-a6fd-bc36f3130832 None admin] RBAC: Authorizing `identity:get_project(project_id=9549a5efb25e493d9a9b245fcc802166)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.613243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8208d5b-4c31-4a5e-a6fd-bc36f3130832 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.614883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 232/448] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/projects/9549a5efb25e493d9a9b245fcc802166 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:52.615634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8208d5b-4c31-4a5e-a6fd-bc36f3130832 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.665431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 217/449] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:52.666273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01485d6-6db6-45c1-a09a-0cc3251e5ee7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.725647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8208d5b-4c31-4a5e-a6fd-bc36f3130832 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.730672 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.760821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939cd37-dec7-4117-82f1-b19f087812e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.761335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939cd37-dec7-4117-82f1-b19f087812e7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.761612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939cd37-dec7-4117-82f1-b19f087812e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.762086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939cd37-dec7-4117-82f1-b19f087812e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.762760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939cd37-dec7-4117-82f1-b19f087812e7 None admin] RBAC: Adding query filter params (group.id=c06e2d3d5b06411c8cccc3da5822ce48, role.id=16317a22e9c94a8886f02efb75ab456e, scope.project.id=79572ca8d9504396b675929feb75e9f2) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:52.763060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939cd37-dec7-4117-82f1-b19f087812e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.764468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939cd37-dec7-4117-82f1-b19f087812e7 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.769229 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01485d6-6db6-45c1-a09a-0cc3251e5ee7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.770153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 233/450] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/role_assignments?group.id=c06e2d3d5b06411c8cccc3da5822ce48&scope.project.id=79572ca8d9504396b675929feb75e9f2&role.id=16317a22e9c94a8886f02efb75ab456e => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:52.770919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939cd37-dec7-4117-82f1-b19f087812e7 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.787687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.822711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac56e4-3321-43fe-9ca2-56cdf57235e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.823789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac56e4-3321-43fe-9ca2-56cdf57235e7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.824240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac56e4-3321-43fe-9ca2-56cdf57235e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.824550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac56e4-3321-43fe-9ca2-56cdf57235e7 None admin] PATH_INFO: `/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.825988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac56e4-3321-43fe-9ca2-56cdf57235e7 None admin] RBAC: Authorizing `identity:get_group(group_id=c06e2d3d5b06411c8cccc3da5822ce48)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.827519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac56e4-3321-43fe-9ca2-56cdf57235e7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.829390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 218/451] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/groups/c06e2d3d5b06411c8cccc3da5822ce48 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:52.835334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac56e4-3321-43fe-9ca2-56cdf57235e7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.893177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939cd37-dec7-4117-82f1-b19f087812e7 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.938597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70b3b8-a86c-4b33-b26a-5f832bba849c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.939239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70b3b8-a86c-4b33-b26a-5f832bba849c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.939520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70b3b8-a86c-4b33-b26a-5f832bba849c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.939794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd70b3b8-a86c-4b33-b26a-5f832bba849c None admin] PATH_INFO: `/v3/projects/66ac24841ce6483cbf8285ea30e957f8` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.941072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70b3b8-a86c-4b33-b26a-5f832bba849c None admin] RBAC: Authorizing `identity:get_project(project_id=66ac24841ce6483cbf8285ea30e957f8)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.942640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd70b3b8-a86c-4b33-b26a-5f832bba849c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.944347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 234/452] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/projects/66ac24841ce6483cbf8285ea30e957f8 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:52.951434 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70b3b8-a86c-4b33-b26a-5f832bba849c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.953922 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac56e4-3321-43fe-9ca2-56cdf57235e7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.994206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:53.002030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea38d4-1d36-4a2c-a103-7bcc967e085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.002820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea38d4-1d36-4a2c-a103-7bcc967e085f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.003262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea38d4-1d36-4a2c-a103-7bcc967e085f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.003578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea38d4-1d36-4a2c-a103-7bcc967e085f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.004463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea38d4-1d36-4a2c-a103-7bcc967e085f None admin] RBAC: Adding query filter params (group.id=c06e2d3d5b06411c8cccc3da5822ce48, role.id=16317a22e9c94a8886f02efb75ab456e, scope.project.id=9549a5efb25e493d9a9b245fcc802166) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:53.004805 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea38d4-1d36-4a2c-a103-7bcc967e085f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.006275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea38d4-1d36-4a2c-a103-7bcc967e085f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.013829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 219/453] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/role_assignments?group.id=c06e2d3d5b06411c8cccc3da5822ce48&scope.project.id=9549a5efb25e493d9a9b245fcc802166&role.id=16317a22e9c94a8886f02efb75ab456e => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:53.058785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:53.082222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd70b3b8-a86c-4b33-b26a-5f832bba849c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.127227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b90423-d6bc-454e-bb1b-de1b04792614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.127923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b90423-d6bc-454e-bb1b-de1b04792614 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.128206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b90423-d6bc-454e-bb1b-de1b04792614 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.128480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b90423-d6bc-454e-bb1b-de1b04792614 None admin] PATH_INFO: `/v3/groups/5ca37afbf4824d64883fd2a018489ff6` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.129846 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b90423-d6bc-454e-bb1b-de1b04792614 None admin] RBAC: Authorizing `identity:get_group(group_id=5ca37afbf4824d64883fd2a018489ff6)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.131330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b90423-d6bc-454e-bb1b-de1b04792614 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.133253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 235/454] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/groups/5ca37afbf4824d64883fd2a018489ff6 => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:53.139438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea38d4-1d36-4a2c-a103-7bcc967e085f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.247991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea38d4-1d36-4a2c-a103-7bcc967e085f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.282017 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739439c7-5b86-4fbf-9c71-541efd3ddcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.282539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739439c7-5b86-4fbf-9c71-541efd3ddcd7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.282817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739439c7-5b86-4fbf-9c71-541efd3ddcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.283079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739439c7-5b86-4fbf-9c71-541efd3ddcd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.283835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739439c7-5b86-4fbf-9c71-541efd3ddcd7 None admin] RBAC: Adding query filter params (group.id=5ca37afbf4824d64883fd2a018489ff6, role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=66ac24841ce6483cbf8285ea30e957f8) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:53.284138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739439c7-5b86-4fbf-9c71-541efd3ddcd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.285632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739439c7-5b86-4fbf-9c71-541efd3ddcd7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.291520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 220/455] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:53 2022] GET /identity/v3/role_assignments?group.id=5ca37afbf4824d64883fd2a018489ff6&scope.project.id=66ac24841ce6483cbf8285ea30e957f8&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:55.570990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a752468-48bc-40c8-8480-98526d6f2460 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.572033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a752468-48bc-40c8-8480-98526d6f2460 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:55.572525 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a752468-48bc-40c8-8480-98526d6f2460 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.574744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 236/456] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:55.579977 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f3a85-081b-496d-ba9f-e05de98a7e13 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.580871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f3a85-081b-496d-ba9f-e05de98a7e13 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:55.581338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f3a85-081b-496d-ba9f-e05de98a7e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.606075 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-112f3a85-081b-496d-ba9f-e05de98a7e13 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:55.607283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-112f3a85-081b-496d-ba9f-e05de98a7e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:55.684146 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:55.706979 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 221/457] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:55.979179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47c152-8be0-4365-817d-b7b587a21168 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.979465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47c152-8be0-4365-817d-b7b587a21168 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:55.979737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47c152-8be0-4365-817d-b7b587a21168 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.981305 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 237/458] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:55.999504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa250-5268-4b01-a1bf-e40c42964cda None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.999792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa250-5268-4b01-a1bf-e40c42964cda None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.000068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa250-5268-4b01-a1bf-e40c42964cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.028812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-402aa250-5268-4b01-a1bf-e40c42964cda None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:56.030012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-402aa250-5268-4b01-a1bf-e40c42964cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:56.035055 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5699b2e-8660-4e62-8c5f-8df18a0b7514 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.035337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5699b2e-8660-4e62-8c5f-8df18a0b7514 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.035609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5699b2e-8660-4e62-8c5f-8df18a0b7514 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.037256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 238/459] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:56.051610 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c9274-e2ac-4d57-891d-d30e05c49243 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.051897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c9274-e2ac-4d57-891d-d30e05c49243 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.052173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c9274-e2ac-4d57-891d-d30e05c49243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.087060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e57c9274-e2ac-4d57-891d-d30e05c49243 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:56.089136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e57c9274-e2ac-4d57-891d-d30e05c49243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:56.119391 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 222/460] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:56.120387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa250-5268-4b01-a1bf-e40c42964cda None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.172349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 239/461] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:56.273983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa250-5268-4b01-a1bf-e40c42964cda None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.288539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc7364-528f-4aa2-b063-be244beb88ff None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.288864 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc7364-528f-4aa2-b063-be244beb88ff None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.289150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc7364-528f-4aa2-b063-be244beb88ff None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.294826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 240/462] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:56.296028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8be3c-eae4-4942-bc86-94d8ae3c0e25 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.296318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8be3c-eae4-4942-bc86-94d8ae3c0e25 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.296591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8be3c-eae4-4942-bc86-94d8ae3c0e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.298311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:56.315128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f744259-ef71-431c-9fb2-94942a118de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.315987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f744259-ef71-431c-9fb2-94942a118de2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.316436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f744259-ef71-431c-9fb2-94942a118de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.316880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f744259-ef71-431c-9fb2-94942a118de2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.329375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f744259-ef71-431c-9fb2-94942a118de2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.331082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f744259-ef71-431c-9fb2-94942a118de2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.333165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-34e8be3c-eae4-4942-bc86-94d8ae3c0e25 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:56.334374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-34e8be3c-eae4-4942-bc86-94d8ae3c0e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-0f744259-ef71-431c-9fb2-94942a118de2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:56.350864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:56.352034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:56.353161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:56.353161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:56.353161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:56.355068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 223/463] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:56.365742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f744259-ef71-431c-9fb2-94942a118de2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.465069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 241/464] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:56.465982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8be3c-eae4-4942-bc86-94d8ae3c0e25 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.487432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:56.549366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f744259-ef71-431c-9fb2-94942a118de2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.594551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f90ae-7f92-4e41-8071-f331819a96ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.595226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f90ae-7f92-4e41-8071-f331819a96ef None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.595417 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f90ae-7f92-4e41-8071-f331819a96ef None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.595855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f90ae-7f92-4e41-8071-f331819a96ef None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.596678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f90ae-7f92-4e41-8071-f331819a96ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:56.597174 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f90ae-7f92-4e41-8071-f331819a96ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.598826 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f90ae-7f92-4e41-8071-f331819a96ef None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.621245 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8be3c-eae4-4942-bc86-94d8ae3c0e25 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.633934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 224/465] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:56.633934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f90ae-7f92-4e41-8071-f331819a96ef None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.664808 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ef4ca-c0c3-4ee8-83a1-a1a477769486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.665772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ef4ca-c0c3-4ee8-83a1-a1a477769486 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.666264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ef4ca-c0c3-4ee8-83a1-a1a477769486 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.666741 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ef4ca-c0c3-4ee8-83a1-a1a477769486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.673079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:56.678390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ef4ca-c0c3-4ee8-83a1-a1a477769486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.680165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ef4ca-c0c3-4ee8-83a1-a1a477769486 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-d45ef4ca-c0c3-4ee8-83a1-a1a477769486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:56.696366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:56.697471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:56.698307 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:56.698307 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:56.698307 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:56.700387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 242/466] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:56.701466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ef4ca-c0c3-4ee8-83a1-a1a477769486 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.754365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f90ae-7f92-4e41-8071-f331819a96ef None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.768901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:56.796840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf355b-c058-418e-873e-e73422abe3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.797819 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf355b-c058-418e-873e-e73422abe3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.798690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf355b-c058-418e-873e-e73422abe3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.799131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf355b-c058-418e-873e-e73422abe3a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.810733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ef4ca-c0c3-4ee8-83a1-a1a477769486 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.813742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf355b-c058-418e-873e-e73422abe3a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.815391 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf355b-c058-418e-873e-e73422abe3a4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-27cf355b-c058-418e-873e-e73422abe3a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:56.835786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:56.836738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:56.837608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:56.837608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:56.837608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:56.838287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 225/467] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:56.839046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf355b-c058-418e-873e-e73422abe3a4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.850027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3e320-148a-45d0-b75d-c0e163a4d2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.850646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3e320-148a-45d0-b75d-c0e163a4d2b5 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.850928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3e320-148a-45d0-b75d-c0e163a4d2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.851203 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3e320-148a-45d0-b75d-c0e163a4d2b5 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.852209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3e320-148a-45d0-b75d-c0e163a4d2b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.854014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3e320-148a-45d0-b75d-c0e163a4d2b5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.879998 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 243/468] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:56 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:56.880918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3e320-148a-45d0-b75d-c0e163a4d2b5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.939777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:56.960860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf355b-c058-418e-873e-e73422abe3a4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:57.008539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bfc4e-21ce-4a28-adf3-fac60fa9c1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:57.009222 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bfc4e-21ce-4a28-adf3-fac60fa9c1dc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.009499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bfc4e-21ce-4a28-adf3-fac60fa9c1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.009814 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bfc4e-21ce-4a28-adf3-fac60fa9c1dc None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.010474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bfc4e-21ce-4a28-adf3-fac60fa9c1dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:57.010771 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bfc4e-21ce-4a28-adf3-fac60fa9c1dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:57.012755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bfc4e-21ce-4a28-adf3-fac60fa9c1dc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:57.032298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 226/469] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:57.033342 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bfc4e-21ce-4a28-adf3-fac60fa9c1dc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:57.060093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3e320-148a-45d0-b75d-c0e163a4d2b5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:57.083265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:57.109002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86c09c-02be-4273-a64e-8ed16f19b3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:57.109985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86c09c-02be-4273-a64e-8ed16f19b3ec None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.110262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86c09c-02be-4273-a64e-8ed16f19b3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.110528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86c09c-02be-4273-a64e-8ed16f19b3ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.123856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86c09c-02be-4273-a64e-8ed16f19b3ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:57.125400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86c09c-02be-4273-a64e-8ed16f19b3ec None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-8d86c09c-02be-4273-a64e-8ed16f19b3ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:57.146166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:57.147184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:57.148530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:57.148530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:57.148530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:57.148530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 244/470] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:57.148530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86c09c-02be-4273-a64e-8ed16f19b3ec None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:57.170692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bfc4e-21ce-4a28-adf3-fac60fa9c1dc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:57.219619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f05e94-2e12-4ad4-aafb-ab36b14b7e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:57.220234 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f05e94-2e12-4ad4-aafb-ab36b14b7e5a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.220515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f05e94-2e12-4ad4-aafb-ab36b14b7e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.220820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f05e94-2e12-4ad4-aafb-ab36b14b7e5a None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.221472 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f05e94-2e12-4ad4-aafb-ab36b14b7e5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:57.221773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f05e94-2e12-4ad4-aafb-ab36b14b7e5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:57.223722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f05e94-2e12-4ad4-aafb-ab36b14b7e5a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:57.237929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 227/471] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:57.239036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f05e94-2e12-4ad4-aafb-ab36b14b7e5a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:57.244605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:57.285294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86c09c-02be-4273-a64e-8ed16f19b3ec None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:57.322576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02b80-97f1-4eaa-9d1b-331b2bfe56a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:57.323174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02b80-97f1-4eaa-9d1b-331b2bfe56a6 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.323446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02b80-97f1-4eaa-9d1b-331b2bfe56a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.323711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e02b80-97f1-4eaa-9d1b-331b2bfe56a6 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.324471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02b80-97f1-4eaa-9d1b-331b2bfe56a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:57.325999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e02b80-97f1-4eaa-9d1b-331b2bfe56a6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:57.357442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 245/472] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:57 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:57.357442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02b80-97f1-4eaa-9d1b-331b2bfe56a6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:57.363151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f05e94-2e12-4ad4-aafb-ab36b14b7e5a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:57.387950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:57.407105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece60769-1fd9-4c51-ae21-252a4c3938df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:57.408145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece60769-1fd9-4c51-ae21-252a4c3938df None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.408616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece60769-1fd9-4c51-ae21-252a4c3938df None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.409145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece60769-1fd9-4c51-ae21-252a4c3938df None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.410058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece60769-1fd9-4c51-ae21-252a4c3938df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:57.410554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece60769-1fd9-4c51-ae21-252a4c3938df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:57.412342 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece60769-1fd9-4c51-ae21-252a4c3938df None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:57.427584 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 228/473] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:57.429340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2c670-d703-42d4-be82-ae206f14b938 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.429849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2c670-d703-42d4-be82-ae206f14b938 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.430304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2c670-d703-42d4-be82-ae206f14b938 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.436839 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 229/474] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:57.438205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2c670-d703-42d4-be82-ae206f14b938 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:57.471652 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e02b80-97f1-4eaa-9d1b-331b2bfe56a6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:57.517018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:57.517445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f313d-a90f-4d51-b036-19b44c1c3630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:57.518190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f313d-a90f-4d51-b036-19b44c1c3630 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.518567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f313d-a90f-4d51-b036-19b44c1c3630 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.518724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f313d-a90f-4d51-b036-19b44c1c3630 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.519380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f313d-a90f-4d51-b036-19b44c1c3630 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:57.521196 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f313d-a90f-4d51-b036-19b44c1c3630 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:57.549752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 246/475] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:57 2022] POST /identity/v3/users => generated 248 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:57.550966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc082cd-74b9-4107-bb74-766a63f88f73 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.551259 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc082cd-74b9-4107-bb74-766a63f88f73 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.551539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc082cd-74b9-4107-bb74-766a63f88f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.563848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2c670-d703-42d4-be82-ae206f14b938 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:57.573955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-bfc082cd-74b9-4107-bb74-766a63f88f73 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:57.575540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bfc082cd-74b9-4107-bb74-766a63f88f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:57.614695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58abc0-4b24-4ed7-ba50-94f9f23fe3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:57.614695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58abc0-4b24-4ed7-ba50-94f9f23fe3ec None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.614695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58abc0-4b24-4ed7-ba50-94f9f23fe3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.615355 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58abc0-4b24-4ed7-ba50-94f9f23fe3ec None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.615608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58abc0-4b24-4ed7-ba50-94f9f23fe3ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:57.617480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58abc0-4b24-4ed7-ba50-94f9f23fe3ec None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:57.651576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 230/476] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:57 2022] POST /identity/v3/users => generated 249 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:57.665182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 247/477] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:57.938914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58abc0-4b24-4ed7-ba50-94f9f23fe3ec None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:58.049370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58abc0-4b24-4ed7-ba50-94f9f23fe3ec None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:58.074938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:58.096781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0b9d2-7dc2-47b3-86bd-7a3177b8fc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:58.097399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0b9d2-7dc2-47b3-86bd-7a3177b8fc7e None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:58.097677 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0b9d2-7dc2-47b3-86bd-7a3177b8fc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:58.097948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0b9d2-7dc2-47b3-86bd-7a3177b8fc7e None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:58.098630 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0b9d2-7dc2-47b3-86bd-7a3177b8fc7e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:58.100038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0b9d2-7dc2-47b3-86bd-7a3177b8fc7e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:58.109747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.sql.core [None req-3dc0b9d2-7dc2-47b3-86bd-7a3177b8fc7e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 12:19:58.109747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 12:19:58.109747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [parameters: {'id': 'e912f33b0eed4cfdbefac9aa91693f40', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 12:19:58.109747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81634) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3dc0b9d2-7dc2-47b3-86bd-7a3177b8fc7e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result.read() Aug 24 12:19:58.126087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:19:58.127010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application yield resource Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise self.value Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application session.commit() Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 12:19:58.127896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.session.flush() Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application compat.raise_( Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 12:19:58.128761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:19:58.129642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result.read() Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:19:58.130427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [parameters: {'id': 'e912f33b0eed4cfdbefac9aa91693f40', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:58.131184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 12:19:58.132025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:19:58.138654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 231/478] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 24 12:19:57 2022] POST /identity/v3/roles => generated 144 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:58.138790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc082cd-74b9-4107-bb74-766a63f88f73 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:58.159384 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:58.280764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc082cd-74b9-4107-bb74-766a63f88f73 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:58.322734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48250722-db13-4647-a62d-ae3a15453beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:58.323361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48250722-db13-4647-a62d-ae3a15453beb None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:58.323634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48250722-db13-4647-a62d-ae3a15453beb None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:58.323903 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48250722-db13-4647-a62d-ae3a15453beb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:58.334726 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48250722-db13-4647-a62d-ae3a15453beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-48250722-db13-4647-a62d-ae3a15453beb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:58.339958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:58.342037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:19:58.343798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 248/479] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:58.347743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0b9d2-7dc2-47b3-86bd-7a3177b8fc7e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:58.495504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0b9d2-7dc2-47b3-86bd-7a3177b8fc7e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:58.530022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be35d2-7000-4557-92a9-551e40674543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:58.530633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be35d2-7000-4557-92a9-551e40674543 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:58.530964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be35d2-7000-4557-92a9-551e40674543 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:58.531183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be35d2-7000-4557-92a9-551e40674543 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:58.531857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be35d2-7000-4557-92a9-551e40674543 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:58.532158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be35d2-7000-4557-92a9-551e40674543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:58.533641 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be35d2-7000-4557-92a9-551e40674543 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:58.542482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 232/480] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:59.147598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e011606-d213-4bbb-9430-c9d3b2ad4ef7 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.148256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e011606-d213-4bbb-9430-c9d3b2ad4ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.148256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e011606-d213-4bbb-9430-c9d3b2ad4ef7 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.149821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 249/481] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:59.155662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2d3b5-56fc-4fb4-a317-8bbb5b3af285 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.156232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2d3b5-56fc-4fb4-a317-8bbb5b3af285 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.156232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2d3b5-56fc-4fb4-a317-8bbb5b3af285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.181615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-0bb2d3b5-56fc-4fb4-a317-8bbb5b3af285 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:59.182795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0bb2d3b5-56fc-4fb4-a317-8bbb5b3af285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:59.242906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:59.279494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 233/482] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:59.777819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717481a7-94d5-4531-9a37-c422fb797047 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.778239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717481a7-94d5-4531-9a37-c422fb797047 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.778454 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717481a7-94d5-4531-9a37-c422fb797047 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.780588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 250/483] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:59.786299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb262ae4-6ad5-4b64-bb2e-515b48320e50 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.786583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb262ae4-6ad5-4b64-bb2e-515b48320e50 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.786855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb262ae4-6ad5-4b64-bb2e-515b48320e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.792166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717481a7-94d5-4531-9a37-c422fb797047 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:59.827077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-cb262ae4-6ad5-4b64-bb2e-515b48320e50 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:59.828280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cb262ae4-6ad5-4b64-bb2e-515b48320e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:59.921150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:59.954518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 234/484] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:59.957280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a4169-b6f1-4ae4-bc0a-25fd1347ec57 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.957574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a4169-b6f1-4ae4-bc0a-25fd1347ec57 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.957851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a4169-b6f1-4ae4-bc0a-25fd1347ec57 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.959409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 235/485] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:19:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:59.968133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fb464-cd66-478f-b867-8a4f8f9a2396 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.968411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fb464-cd66-478f-b867-8a4f8f9a2396 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.968695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fb464-cd66-478f-b867-8a4f8f9a2396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.972739 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717481a7-94d5-4531-9a37-c422fb797047 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.005472 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ec9fb464-cd66-478f-b867-8a4f8f9a2396 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:00.006653 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ec9fb464-cd66-478f-b867-8a4f8f9a2396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:00.037366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3f00a-308f-47aa-a367-c7147cc7d68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.037914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3f00a-308f-47aa-a367-c7147cc7d68b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.038188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3f00a-308f-47aa-a367-c7147cc7d68b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.038451 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3f00a-308f-47aa-a367-c7147cc7d68b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.056607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3f00a-308f-47aa-a367-c7147cc7d68b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-a2b3f00a-308f-47aa-a367-c7147cc7d68b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:00.061950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:00.062807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:00.063597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 251/486] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:19:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:00.064467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b03492-a23b-4833-978d-4680bd50a5d5 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.064908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b03492-a23b-4833-978d-4680bd50a5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.065334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b03492-a23b-4833-978d-4680bd50a5d5 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.066884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 252/487] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:00.070676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b03492-a23b-4833-978d-4680bd50a5d5 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.120474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 236/488] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:00.122240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ac04f-e86f-4cbc-a4f1-451a98af7ca6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.122900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ac04f-e86f-4cbc-a4f1-451a98af7ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.123367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ac04f-e86f-4cbc-a4f1-451a98af7ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.146221 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-270ac04f-e86f-4cbc-a4f1-451a98af7ca6 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:00.147866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-270ac04f-e86f-4cbc-a4f1-451a98af7ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:00.180737 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.222071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b03492-a23b-4833-978d-4680bd50a5d5 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.225582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 237/489] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:00.261169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.266971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d6743-becb-4cf2-9bbd-0e90b5e2ca8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.267586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d6743-becb-4cf2-9bbd-0e90b5e2ca8b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.267862 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d6743-becb-4cf2-9bbd-0e90b5e2ca8b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.268132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d6743-becb-4cf2-9bbd-0e90b5e2ca8b None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.268824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d6743-becb-4cf2-9bbd-0e90b5e2ca8b None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:00.269124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d6743-becb-4cf2-9bbd-0e90b5e2ca8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.270980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d6743-becb-4cf2-9bbd-0e90b5e2ca8b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.284752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 253/490] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:00.290362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ac04f-e86f-4cbc-a4f1-451a98af7ca6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.298739 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d6743-becb-4cf2-9bbd-0e90b5e2ca8b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.416524 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d6743-becb-4cf2-9bbd-0e90b5e2ca8b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.452515 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816e5d-f5a8-4742-9be0-00cb1a66a1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.453432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816e5d-f5a8-4742-9be0-00cb1a66a1eb None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.453731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ac04f-e86f-4cbc-a4f1-451a98af7ca6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.454038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816e5d-f5a8-4742-9be0-00cb1a66a1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.454443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816e5d-f5a8-4742-9be0-00cb1a66a1eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.465988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68816e5d-f5a8-4742-9be0-00cb1a66a1eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-68816e5d-f5a8-4742-9be0-00cb1a66a1eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:00.472941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:00.474051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:00.475139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:00.475139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:00.475139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:00.475139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:00.475139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:00.475139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:00.475139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:00.475139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:00.475139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:00.475139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:00.475139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:00.475139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:00.477307 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 254/491] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:00.478431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816e5d-f5a8-4742-9be0-00cb1a66a1eb None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.489781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d9b47-fb47-4456-bc21-41092c968bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.490456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d9b47-fb47-4456-bc21-41092c968bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.490736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d9b47-fb47-4456-bc21-41092c968bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.491001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d9b47-fb47-4456-bc21-41092c968bb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.502226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d9b47-fb47-4456-bc21-41092c968bb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.503853 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d9b47-fb47-4456-bc21-41092c968bb0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d33d9b47-fb47-4456-bc21-41092c968bb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:00.519284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:00.520398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:00.521409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:00.521409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:00.521409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:00.521409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 238/492] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:00.521616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d9b47-fb47-4456-bc21-41092c968bb0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.542551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.546870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.593976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816e5d-f5a8-4742-9be0-00cb1a66a1eb None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.631229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f309ab-19af-4444-8e66-9e28cba6ce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.632199 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f309ab-19af-4444-8e66-9e28cba6ce8d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.632636 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f309ab-19af-4444-8e66-9e28cba6ce8d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.633058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f309ab-19af-4444-8e66-9e28cba6ce8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.634649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d9b47-fb47-4456-bc21-41092c968bb0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.644342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f309ab-19af-4444-8e66-9e28cba6ce8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-98f309ab-19af-4444-8e66-9e28cba6ce8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:00.650335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:00.651338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:00.652336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:00.652336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:00.652336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:00.652336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:00.652336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:00.652336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:00.652336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:00.652336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:00.652336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:00.652336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:00.652336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:00.652336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:00.654605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 255/493] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:00.655561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f309ab-19af-4444-8e66-9e28cba6ce8d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.672297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aace008-c3e5-430a-9d65-d024dd58522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.673031 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aace008-c3e5-430a-9d65-d024dd58522d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.673318 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aace008-c3e5-430a-9d65-d024dd58522d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.673582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aace008-c3e5-430a-9d65-d024dd58522d None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.674252 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aace008-c3e5-430a-9d65-d024dd58522d None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:00.674550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aace008-c3e5-430a-9d65-d024dd58522d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.675928 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aace008-c3e5-430a-9d65-d024dd58522d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.684281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 239/494] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:00.685302 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aace008-c3e5-430a-9d65-d024dd58522d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.774626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f309ab-19af-4444-8e66-9e28cba6ce8d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.776297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.792494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.802625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aace008-c3e5-430a-9d65-d024dd58522d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.816554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122610b-efc7-40f4-8d56-6707317ad2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.817323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122610b-efc7-40f4-8d56-6707317ad2ea None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.817633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122610b-efc7-40f4-8d56-6707317ad2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.817945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122610b-efc7-40f4-8d56-6707317ad2ea None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.818588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122610b-efc7-40f4-8d56-6707317ad2ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:00.818887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122610b-efc7-40f4-8d56-6707317ad2ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.820384 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122610b-efc7-40f4-8d56-6707317ad2ea None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.832676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 256/495] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:00.833313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122610b-efc7-40f4-8d56-6707317ad2ea None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.851654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc848a79-f46e-4960-a568-1d2198abdc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.852993 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc848a79-f46e-4960-a568-1d2198abdc46 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.853466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc848a79-f46e-4960-a568-1d2198abdc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.853902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc848a79-f46e-4960-a568-1d2198abdc46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.865353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc848a79-f46e-4960-a568-1d2198abdc46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-bc848a79-f46e-4960-a568-1d2198abdc46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:00.870666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:00.871784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:00.872565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 240/496] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:00.873283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc848a79-f46e-4960-a568-1d2198abdc46 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.946226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122610b-efc7-40f4-8d56-6707317ad2ea None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.989627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c63b17-0938-41b9-811a-51d9e4010aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.990243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63b17-0938-41b9-811a-51d9e4010aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.990518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63b17-0938-41b9-811a-51d9e4010aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.990779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63b17-0938-41b9-811a-51d9e4010aa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.991441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c63b17-0938-41b9-811a-51d9e4010aa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:00.991735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c63b17-0938-41b9-811a-51d9e4010aa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.993193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c63b17-0938-41b9-811a-51d9e4010aa7 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.993307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc848a79-f46e-4960-a568-1d2198abdc46 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.001771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 257/497] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:01.002604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c63b17-0938-41b9-811a-51d9e4010aa7 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.035217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbc8b7-77cb-4d6c-8cad-9e1878459a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.036132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbc8b7-77cb-4d6c-8cad-9e1878459a0b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.036459 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbc8b7-77cb-4d6c-8cad-9e1878459a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.036737 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbc8b7-77cb-4d6c-8cad-9e1878459a0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.039826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.048340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.051225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbc8b7-77cb-4d6c-8cad-9e1878459a0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.052883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbc8b7-77cb-4d6c-8cad-9e1878459a0b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3bfbc8b7-77cb-4d6c-8cad-9e1878459a0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:01.069376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:01.070319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:01.071154 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:01.071154 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:01.071154 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:01.071154 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 241/498] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:01.071905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbc8b7-77cb-4d6c-8cad-9e1878459a0b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.112854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c63b17-0938-41b9-811a-51d9e4010aa7 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.149804 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa84cb-752f-46ba-971a-fb90c65ead3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.150431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa84cb-752f-46ba-971a-fb90c65ead3d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.150706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa84cb-752f-46ba-971a-fb90c65ead3d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.151014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa84cb-752f-46ba-971a-fb90c65ead3d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.158824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa84cb-752f-46ba-971a-fb90c65ead3d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.160286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa84cb-752f-46ba-971a-fb90c65ead3d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-13aa84cb-752f-46ba-971a-fb90c65ead3d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:01.174026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:01.174937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:01.175798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:01.175798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:01.175798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:01.175798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:01.175798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:01.175798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:01.175798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:01.175798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:01.175798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:01.175798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:01.176293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbc8b7-77cb-4d6c-8cad-9e1878459a0b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.176363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 258/499] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:01.176363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa84cb-752f-46ba-971a-fb90c65ead3d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.213554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e1deb-83e5-437f-8748-f818e2cbcfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.214474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e1deb-83e5-437f-8748-f818e2cbcfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.214821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e1deb-83e5-437f-8748-f818e2cbcfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.215247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e1deb-83e5-437f-8748-f818e2cbcfa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.216094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e1deb-83e5-437f-8748-f818e2cbcfa6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:01.216399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e1deb-83e5-437f-8748-f818e2cbcfa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.217858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e1deb-83e5-437f-8748-f818e2cbcfa6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.231092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 242/500] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:01.231092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e1deb-83e5-437f-8748-f818e2cbcfa6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.273168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.279864 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa84cb-752f-46ba-971a-fb90c65ead3d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.281473 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.312392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef270b3c-40aa-4149-82f9-9140fb4018ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.312957 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef270b3c-40aa-4149-82f9-9140fb4018ec None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.313228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef270b3c-40aa-4149-82f9-9140fb4018ec None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.313497 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef270b3c-40aa-4149-82f9-9140fb4018ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.324090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef270b3c-40aa-4149-82f9-9140fb4018ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.325616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef270b3c-40aa-4149-82f9-9140fb4018ec None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.327318 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e1deb-83e5-437f-8748-f818e2cbcfa6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-ef270b3c-40aa-4149-82f9-9140fb4018ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:01.340211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:01.341276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:01.342123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:01.342123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:01.342123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:01.342123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 259/501] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:01.342409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd84f3-520d-4e64-a74d-8e5e77126de4 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.342678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd84f3-520d-4e64-a74d-8e5e77126de4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.342956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd84f3-520d-4e64-a74d-8e5e77126de4 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.344175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 260/502] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:01.344922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd84f3-520d-4e64-a74d-8e5e77126de4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.364288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d1f4d-087d-48f6-957b-02c3f951f315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.364925 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d1f4d-087d-48f6-957b-02c3f951f315 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.365199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d1f4d-087d-48f6-957b-02c3f951f315 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.365466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d1f4d-087d-48f6-957b-02c3f951f315 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.366107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d1f4d-087d-48f6-957b-02c3f951f315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:01.366405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d1f4d-087d-48f6-957b-02c3f951f315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.367778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d1f4d-087d-48f6-957b-02c3f951f315 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.379832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 243/503] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:01.380671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d1f4d-087d-48f6-957b-02c3f951f315 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.446274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd84f3-520d-4e64-a74d-8e5e77126de4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.479889 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f25ca-5d27-4db6-90f4-bb9200ec4213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.482442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f25ca-5d27-4db6-90f4-bb9200ec4213 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.482631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d1f4d-087d-48f6-957b-02c3f951f315 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.482730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f25ca-5d27-4db6-90f4-bb9200ec4213 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.482989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f25ca-5d27-4db6-90f4-bb9200ec4213 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.483731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f25ca-5d27-4db6-90f4-bb9200ec4213 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:01.484126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f25ca-5d27-4db6-90f4-bb9200ec4213 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.485500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f25ca-5d27-4db6-90f4-bb9200ec4213 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.498126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.519785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e97e95-1375-4af3-b4b8-d1f13e20aaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.520394 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e97e95-1375-4af3-b4b8-d1f13e20aaee None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.520690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e97e95-1375-4af3-b4b8-d1f13e20aaee None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.520962 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e97e95-1375-4af3-b4b8-d1f13e20aaee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.522371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 261/504] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:01.523395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f25ca-5d27-4db6-90f4-bb9200ec4213 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.531703 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e97e95-1375-4af3-b4b8-d1f13e20aaee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.533332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e97e95-1375-4af3-b4b8-d1f13e20aaee None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.533449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-89e97e95-1375-4af3-b4b8-d1f13e20aaee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:01.548906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:01.549995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:01.551027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:01.551027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:01.551027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:01.552418 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 244/505] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:01.553851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d7a8e-ff48-4921-9539-7fee3340e9bf None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.554135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d7a8e-ff48-4921-9539-7fee3340e9bf None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.554402 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d7a8e-ff48-4921-9539-7fee3340e9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.572133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-536d7a8e-ff48-4921-9539-7fee3340e9bf None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:01.573275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-536d7a8e-ff48-4921-9539-7fee3340e9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:01.637429 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f25ca-5d27-4db6-90f4-bb9200ec4213 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.641882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 245/506] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:01.642702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d7a8e-ff48-4921-9539-7fee3340e9bf None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.676744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd114b-0649-4512-9ff6-05d8bad4bb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.677359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd114b-0649-4512-9ff6-05d8bad4bb02 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.677623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd114b-0649-4512-9ff6-05d8bad4bb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.677889 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd114b-0649-4512-9ff6-05d8bad4bb02 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.678520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd114b-0649-4512-9ff6-05d8bad4bb02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:01.678827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd114b-0649-4512-9ff6-05d8bad4bb02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.680213 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd114b-0649-4512-9ff6-05d8bad4bb02 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.693212 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 262/507] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:01.694027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd114b-0649-4512-9ff6-05d8bad4bb02 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.755107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.770092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d7a8e-ff48-4921-9539-7fee3340e9bf None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.773701 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.805101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cf7ac-b43c-4189-b44a-251accd58992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.805729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cf7ac-b43c-4189-b44a-251accd58992 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.806006 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cf7ac-b43c-4189-b44a-251accd58992 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.806277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cf7ac-b43c-4189-b44a-251accd58992 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.812778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd114b-0649-4512-9ff6-05d8bad4bb02 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.813852 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cf7ac-b43c-4189-b44a-251accd58992 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.815387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32cf7ac-b43c-4189-b44a-251accd58992 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f32cf7ac-b43c-4189-b44a-251accd58992 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:01.834085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:01.835055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:01.835973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:01.835973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:01.835973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:01.835973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:01.835973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:01.835973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:01.835973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:01.835973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:01.835973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:01.835973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:01.840157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 246/508] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:01.841469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cf7ac-b43c-4189-b44a-251accd58992 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.856903 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a8663-f656-4e22-abad-492dbfe0175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.857697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a8663-f656-4e22-abad-492dbfe0175e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.858220 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a8663-f656-4e22-abad-492dbfe0175e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.858480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a8663-f656-4e22-abad-492dbfe0175e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.927674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a8663-f656-4e22-abad-492dbfe0175e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.929153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a8663-f656-4e22-abad-492dbfe0175e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.942691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cf7ac-b43c-4189-b44a-251accd58992 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.975873 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f0e2e-e1b5-4c48-a731-2e0589891e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.976465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f0e2e-e1b5-4c48-a731-2e0589891e82 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.976823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f0e2e-e1b5-4c48-a731-2e0589891e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.977097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f0e2e-e1b5-4c48-a731-2e0589891e82 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.977734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f0e2e-e1b5-4c48-a731-2e0589891e82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:01.978062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f0e2e-e1b5-4c48-a731-2e0589891e82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.979455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f0e2e-e1b5-4c48-a731-2e0589891e82 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.991525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 247/509] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:01.992336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f0e2e-e1b5-4c48-a731-2e0589891e82 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.993365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.010045 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.031460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 263/510] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:02.032360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a8663-f656-4e22-abad-492dbfe0175e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.107235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f0e2e-e1b5-4c48-a731-2e0589891e82 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.140115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a8663-f656-4e22-abad-492dbfe0175e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.141926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae5153-71d7-4cc7-9b95-b9e6e0501533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.142572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae5153-71d7-4cc7-9b95-b9e6e0501533 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.142852 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae5153-71d7-4cc7-9b95-b9e6e0501533 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.143119 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae5153-71d7-4cc7-9b95-b9e6e0501533 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.150211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae5153-71d7-4cc7-9b95-b9e6e0501533 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.151680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae5153-71d7-4cc7-9b95-b9e6e0501533 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-01ae5153-71d7-4cc7-9b95-b9e6e0501533 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:02.163593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:02.164469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:02.170972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:02.170972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:02.170972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:02.170972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:02.170972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:02.170972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:02.170972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:02.170972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:02.170972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:20:02.170972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:02.170972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 248/511] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:02.170972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae5153-71d7-4cc7-9b95-b9e6e0501533 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.174252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523694c-f823-49cf-bc27-06e17b6eb8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.174861 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523694c-f823-49cf-bc27-06e17b6eb8cb None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.175134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523694c-f823-49cf-bc27-06e17b6eb8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.175399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523694c-f823-49cf-bc27-06e17b6eb8cb None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.176124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523694c-f823-49cf-bc27-06e17b6eb8cb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:02.176421 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523694c-f823-49cf-bc27-06e17b6eb8cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.177884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523694c-f823-49cf-bc27-06e17b6eb8cb None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.195532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 264/512] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:02.196361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523694c-f823-49cf-bc27-06e17b6eb8cb None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.222371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.241730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.266004 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae5153-71d7-4cc7-9b95-b9e6e0501533 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.299431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523694c-f823-49cf-bc27-06e17b6eb8cb None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.299957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67149c4-95ae-413e-b41b-dbddb7dfa2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.300581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67149c4-95ae-413e-b41b-dbddb7dfa2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.300883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67149c4-95ae-413e-b41b-dbddb7dfa2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.301149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67149c4-95ae-413e-b41b-dbddb7dfa2b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.312060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67149c4-95ae-413e-b41b-dbddb7dfa2b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.313543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67149c4-95ae-413e-b41b-dbddb7dfa2b7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e67149c4-95ae-413e-b41b-dbddb7dfa2b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:02.327731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:02.328726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:02.329568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:02.329568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:02.329568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:02.329568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 249/513] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:02.329791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67149c4-95ae-413e-b41b-dbddb7dfa2b7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.339977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c0718-8d3b-4702-aaf9-a77bc5e0f5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.340609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c0718-8d3b-4702-aaf9-a77bc5e0f5cc None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.340918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c0718-8d3b-4702-aaf9-a77bc5e0f5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.341190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c0718-8d3b-4702-aaf9-a77bc5e0f5cc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.348448 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c0718-8d3b-4702-aaf9-a77bc5e0f5cc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.349953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c0718-8d3b-4702-aaf9-a77bc5e0f5cc None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-871c0718-8d3b-4702-aaf9-a77bc5e0f5cc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:02.361320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:02.362226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:02.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:02.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:02.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:02.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:02.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:02.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:02.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:02.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:02.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:20:02.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:02.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 265/514] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:02.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c0718-8d3b-4702-aaf9-a77bc5e0f5cc None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.440920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67149c4-95ae-413e-b41b-dbddb7dfa2b7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.451290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.468494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c0718-8d3b-4702-aaf9-a77bc5e0f5cc None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.469008 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.479090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f14caf-cdb5-48d0-afaf-7546441200de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.479724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f14caf-cdb5-48d0-afaf-7546441200de None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.479999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f14caf-cdb5-48d0-afaf-7546441200de None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.480260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f14caf-cdb5-48d0-afaf-7546441200de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.491443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f14caf-cdb5-48d0-afaf-7546441200de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.492972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f14caf-cdb5-48d0-afaf-7546441200de None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.505102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418694bb-cb56-455d-bd64-c799ffb56c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.505712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418694bb-cb56-455d-bd64-c799ffb56c5e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.505988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418694bb-cb56-455d-bd64-c799ffb56c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.506258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418694bb-cb56-455d-bd64-c799ffb56c5e None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.506993 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418694bb-cb56-455d-bd64-c799ffb56c5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:02.507295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418694bb-cb56-455d-bd64-c799ffb56c5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-56f14caf-cdb5-48d0-afaf-7546441200de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:02.507578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:02.508577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:02.509535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:02.509535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:02.509535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:02.509535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 250/515] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:02.509746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418694bb-cb56-455d-bd64-c799ffb56c5e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.509820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f14caf-cdb5-48d0-afaf-7546441200de None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.529615 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 266/516] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:02.531784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418694bb-cb56-455d-bd64-c799ffb56c5e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.615893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f14caf-cdb5-48d0-afaf-7546441200de None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.635106 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418694bb-cb56-455d-bd64-c799ffb56c5e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.649388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45daca75-7752-4a7b-92ff-d008a5106c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.649992 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daca75-7752-4a7b-92ff-d008a5106c4f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.650268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daca75-7752-4a7b-92ff-d008a5106c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.650537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daca75-7752-4a7b-92ff-d008a5106c4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.668094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95de79-a7c4-4749-a02f-c9a5f034dee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.668718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95de79-a7c4-4749-a02f-c9a5f034dee8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.669000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95de79-a7c4-4749-a02f-c9a5f034dee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.669271 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95de79-a7c4-4749-a02f-c9a5f034dee8 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.669908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95de79-a7c4-4749-a02f-c9a5f034dee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:02.670209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95de79-a7c4-4749-a02f-c9a5f034dee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.671586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95de79-a7c4-4749-a02f-c9a5f034dee8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.682496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 267/517] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:02.683272 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95de79-a7c4-4749-a02f-c9a5f034dee8 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.688668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.709658 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.731009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45daca75-7752-4a7b-92ff-d008a5106c4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.732508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45daca75-7752-4a7b-92ff-d008a5106c4f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.789176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95de79-a7c4-4749-a02f-c9a5f034dee8 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.822895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaa057-16f0-41b2-b267-9853f446011b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.823501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaa057-16f0-41b2-b267-9853f446011b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.823780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaa057-16f0-41b2-b267-9853f446011b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.824051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaa057-16f0-41b2-b267-9853f446011b None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.824934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaa057-16f0-41b2-b267-9853f446011b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:02.825241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaa057-16f0-41b2-b267-9853f446011b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.826679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaa057-16f0-41b2-b267-9853f446011b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.834357 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 251/518] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:02.835158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45daca75-7752-4a7b-92ff-d008a5106c4f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.844682 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 268/519] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:02.845390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaa057-16f0-41b2-b267-9853f446011b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.907893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.933410 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.949486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaa057-16f0-41b2-b267-9853f446011b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.953059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45daca75-7752-4a7b-92ff-d008a5106c4f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.987205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32061a-5747-4d4d-ab2d-5d06e50d7679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.987634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eef45f-f101-4aff-919f-f42cc8993ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.987814 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32061a-5747-4d4d-ab2d-5d06e50d7679 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.988088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32061a-5747-4d4d-ab2d-5d06e50d7679 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.988237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eef45f-f101-4aff-919f-f42cc8993ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.988354 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32061a-5747-4d4d-ab2d-5d06e50d7679 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.988562 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eef45f-f101-4aff-919f-f42cc8993ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.988858 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eef45f-f101-4aff-919f-f42cc8993ca2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.989035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32061a-5747-4d4d-ab2d-5d06e50d7679 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:02.989332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32061a-5747-4d4d-ab2d-5d06e50d7679 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.990716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32061a-5747-4d4d-ab2d-5d06e50d7679 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.002490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 252/520] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:03.003278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32061a-5747-4d4d-ab2d-5d06e50d7679 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.055900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eef45f-f101-4aff-919f-f42cc8993ca2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.057417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eef45f-f101-4aff-919f-f42cc8993ca2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.106848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32061a-5747-4d4d-ab2d-5d06e50d7679 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.114906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.141373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b21a0-bf50-4d0c-92bb-8ad910318e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.141975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b21a0-bf50-4d0c-92bb-8ad910318e60 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.142247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b21a0-bf50-4d0c-92bb-8ad910318e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.142511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b21a0-bf50-4d0c-92bb-8ad910318e60 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.143141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b21a0-bf50-4d0c-92bb-8ad910318e60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.144594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b21a0-bf50-4d0c-92bb-8ad910318e60 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.155684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.159296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 269/521] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:03.160071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eef45f-f101-4aff-919f-f42cc8993ca2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.168909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 253/522] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:20:03 2022] POST /identity/v3/users => generated 247 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:03.169821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b21a0-bf50-4d0c-92bb-8ad910318e60 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.280863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eef45f-f101-4aff-919f-f42cc8993ca2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.298946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b21a0-bf50-4d0c-92bb-8ad910318e60 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.321651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bd26f-0c03-41ef-ab13-98177f683df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.322287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bd26f-0c03-41ef-ab13-98177f683df4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.322559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bd26f-0c03-41ef-ab13-98177f683df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.322824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bd26f-0c03-41ef-ab13-98177f683df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.333580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bd26f-0c03-41ef-ab13-98177f683df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.335161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bd26f-0c03-41ef-ab13-98177f683df4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.343279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335eb9-b30b-452b-8804-16726d5ec658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.344190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335eb9-b30b-452b-8804-16726d5ec658 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.344472 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335eb9-b30b-452b-8804-16726d5ec658 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.344883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335eb9-b30b-452b-8804-16726d5ec658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-f17bd26f-0c03-41ef-ab13-98177f683df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:03.351054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:03.351922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:03.352787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:03.352787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:03.352787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:03.352787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 270/523] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:03.354574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bd26f-0c03-41ef-ab13-98177f683df4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.373583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.418889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.427707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9335eb9-b30b-452b-8804-16726d5ec658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.429222 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9335eb9-b30b-452b-8804-16726d5ec658 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.466330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bd26f-0c03-41ef-ab13-98177f683df4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.502270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937d0d4-c592-4bad-ba24-4a3451328381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.503012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937d0d4-c592-4bad-ba24-4a3451328381 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.503287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937d0d4-c592-4bad-ba24-4a3451328381 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.503561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937d0d4-c592-4bad-ba24-4a3451328381 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.510486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937d0d4-c592-4bad-ba24-4a3451328381 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.512232 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937d0d4-c592-4bad-ba24-4a3451328381 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-7937d0d4-c592-4bad-ba24-4a3451328381 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:03.525609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:03.526748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:03.527730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:03.527730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:03.527730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:03.527730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:03.527730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:03.527730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:03.527730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:03.527730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:03.531612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 271/524] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:03.532340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937d0d4-c592-4bad-ba24-4a3451328381 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.539462 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 254/525] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:03.540319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335eb9-b30b-452b-8804-16726d5ec658 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.606970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.639657 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937d0d4-c592-4bad-ba24-4a3451328381 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.646619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.654673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335eb9-b30b-452b-8804-16726d5ec658 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.675427 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80655a43-84f2-47de-8772-42cbbf00d696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.675984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80655a43-84f2-47de-8772-42cbbf00d696 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.676263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80655a43-84f2-47de-8772-42cbbf00d696 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.676532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80655a43-84f2-47de-8772-42cbbf00d696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.689071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80655a43-84f2-47de-8772-42cbbf00d696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.690590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80655a43-84f2-47de-8772-42cbbf00d696 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.692251 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e1259-3a96-478d-a361-0c949015ed1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.692883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e1259-3a96-478d-a361-0c949015ed1b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.693160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e1259-3a96-478d-a361-0c949015ed1b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.693576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e1259-3a96-478d-a361-0c949015ed1b None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.694063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e1259-3a96-478d-a361-0c949015ed1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:03.694364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e1259-3a96-478d-a361-0c949015ed1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.695773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e1259-3a96-478d-a361-0c949015ed1b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-80655a43-84f2-47de-8772-42cbbf00d696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:03.705372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:03.706559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:03.707378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:03.707378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:03.707378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:03.707378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 272/526] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:03.707378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80655a43-84f2-47de-8772-42cbbf00d696 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.713865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 255/527] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:03.715405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e1259-3a96-478d-a361-0c949015ed1b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.825015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80655a43-84f2-47de-8772-42cbbf00d696 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.828968 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e1259-3a96-478d-a361-0c949015ed1b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.850894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.862957 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a434d8a-b2a7-4178-8800-604ee2efd9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.863563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a434d8a-b2a7-4178-8800-604ee2efd9ad None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.863840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a434d8a-b2a7-4178-8800-604ee2efd9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.864103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a434d8a-b2a7-4178-8800-604ee2efd9ad None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.864822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a434d8a-b2a7-4178-8800-604ee2efd9ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:03.865125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a434d8a-b2a7-4178-8800-604ee2efd9ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.865706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbdd1c-460e-439a-b5ad-6b569492f550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.866401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbdd1c-460e-439a-b5ad-6b569492f550 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.866562 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a434d8a-b2a7-4178-8800-604ee2efd9ad None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.866789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbdd1c-460e-439a-b5ad-6b569492f550 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.867124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbdd1c-460e-439a-b5ad-6b569492f550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.879183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbdd1c-460e-439a-b5ad-6b569492f550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.880632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbdd1c-460e-439a-b5ad-6b569492f550 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.885282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 273/528] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:03.886024 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a434d8a-b2a7-4178-8800-604ee2efd9ad None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-37dbdd1c-460e-439a-b5ad-6b569492f550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:03.901113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:03.902437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:03.903525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:03.903525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:03.903525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:03.907633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 256/529] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:03.908472 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbdd1c-460e-439a-b5ad-6b569492f550 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.921832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:04.000724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a434d8a-b2a7-4178-8800-604ee2efd9ad None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.025095 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbdd1c-460e-439a-b5ad-6b569492f550 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.039190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff2081-a103-4e4d-9794-93e442d80341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.039811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff2081-a103-4e4d-9794-93e442d80341 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.040094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff2081-a103-4e4d-9794-93e442d80341 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.040366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff2081-a103-4e4d-9794-93e442d80341 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.041045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff2081-a103-4e4d-9794-93e442d80341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:04.041341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff2081-a103-4e4d-9794-93e442d80341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.042749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff2081-a103-4e4d-9794-93e442d80341 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.055151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 274/530] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:04.056271 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff2081-a103-4e4d-9794-93e442d80341 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.067184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b9969-9399-459f-b208-705a4a8a3f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.068466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b9969-9399-459f-b208-705a4a8a3f03 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.068941 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b9969-9399-459f-b208-705a4a8a3f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.069721 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b9969-9399-459f-b208-705a4a8a3f03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.078795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b9969-9399-459f-b208-705a4a8a3f03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.081222 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b9969-9399-459f-b208-705a4a8a3f03 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-966b9969-9399-459f-b208-705a4a8a3f03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:04.096903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:04.097758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:04.098718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:04.098718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:04.098718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:04.098718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:04.098718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:04.098718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:04.098718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:04.098718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:04.100458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 257/531] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:04.101388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b9969-9399-459f-b208-705a4a8a3f03 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.110189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:04.171909 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff2081-a103-4e4d-9794-93e442d80341 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.178106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:04.209635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5f9f1-31bb-4b5d-8ccc-654d9a1bbf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.210250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5f9f1-31bb-4b5d-8ccc-654d9a1bbf9e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.210525 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5f9f1-31bb-4b5d-8ccc-654d9a1bbf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.210794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5f9f1-31bb-4b5d-8ccc-654d9a1bbf9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.211515 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5f9f1-31bb-4b5d-8ccc-654d9a1bbf9e None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=e65026e01df242e7be11ed24214a4efa) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:04.211817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5f9f1-31bb-4b5d-8ccc-654d9a1bbf9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.213297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5f9f1-31bb-4b5d-8ccc-654d9a1bbf9e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.216595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b9969-9399-459f-b208-705a4a8a3f03 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.219804 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 275/532] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/role_assignments?user.id=e65026e01df242e7be11ed24214a4efa&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:04.220509 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5f9f1-31bb-4b5d-8ccc-654d9a1bbf9e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.266422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a13515-48a0-467a-8ab3-c89d5916a6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.267423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a13515-48a0-467a-8ab3-c89d5916a6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.267908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a13515-48a0-467a-8ab3-c89d5916a6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.268356 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a13515-48a0-467a-8ab3-c89d5916a6d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.269277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a13515-48a0-467a-8ab3-c89d5916a6d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:04.269771 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a13515-48a0-467a-8ab3-c89d5916a6d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.271662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a13515-48a0-467a-8ab3-c89d5916a6d3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.287171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 258/533] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:04.287829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a13515-48a0-467a-8ab3-c89d5916a6d3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.363361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5f9f1-31bb-4b5d-8ccc-654d9a1bbf9e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.374235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:04.404483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad69518-6324-41dc-bc74-d441a6d3bde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.405550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad69518-6324-41dc-bc74-d441a6d3bde4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.406067 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad69518-6324-41dc-bc74-d441a6d3bde4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.406526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad69518-6324-41dc-bc74-d441a6d3bde4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.414663 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad69518-6324-41dc-bc74-d441a6d3bde4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.416389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad69518-6324-41dc-bc74-d441a6d3bde4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.422841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a13515-48a0-467a-8ab3-c89d5916a6d3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-0ad69518-6324-41dc-bc74-d441a6d3bde4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:04.429543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:04.430353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:04.431180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:04.431180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:04.431180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:04.431180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:04.431180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:04.431180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:04.431180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:04.431180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:04.437984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 276/534] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:04.439109 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad69518-6324-41dc-bc74-d441a6d3bde4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.465268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaf1ae-a66d-4a2c-b89d-c7feced6fea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.465900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaf1ae-a66d-4a2c-b89d-c7feced6fea4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.466181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaf1ae-a66d-4a2c-b89d-c7feced6fea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.466457 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaf1ae-a66d-4a2c-b89d-c7feced6fea4 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.467591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccaf1ae-a66d-4a2c-b89d-c7feced6fea4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:04.467902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccaf1ae-a66d-4a2c-b89d-c7feced6fea4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.469951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccaf1ae-a66d-4a2c-b89d-c7feced6fea4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.485028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:04.500145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 259/535] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:04.500145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaf1ae-a66d-4a2c-b89d-c7feced6fea4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.554666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad69518-6324-41dc-bc74-d441a6d3bde4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.590362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930ade0-b8f1-4d61-ac42-3f51838ecfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.591417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930ade0-b8f1-4d61-ac42-3f51838ecfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.591905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930ade0-b8f1-4d61-ac42-3f51838ecfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.592365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930ade0-b8f1-4d61-ac42-3f51838ecfb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.601184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930ade0-b8f1-4d61-ac42-3f51838ecfb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.604371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930ade0-b8f1-4d61-ac42-3f51838ecfb3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-7930ade0-b8f1-4d61-ac42-3f51838ecfb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:04.616533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:04.617443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:04.618262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:04.618262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:04.618262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:04.618262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:04.618262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:04.618262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:04.618262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:04.618262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:04.619117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaf1ae-a66d-4a2c-b89d-c7feced6fea4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.621042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 277/536] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:04.622154 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930ade0-b8f1-4d61-ac42-3f51838ecfb3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.652363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:04.660819 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f985ef-a3e7-4773-b95c-3cd3f1d5a7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.661333 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f985ef-a3e7-4773-b95c-3cd3f1d5a7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.661613 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f985ef-a3e7-4773-b95c-3cd3f1d5a7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.661916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f985ef-a3e7-4773-b95c-3cd3f1d5a7c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.662696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f985ef-a3e7-4773-b95c-3cd3f1d5a7c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:04.663000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f985ef-a3e7-4773-b95c-3cd3f1d5a7c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.664575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f985ef-a3e7-4773-b95c-3cd3f1d5a7c8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.683021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 260/537] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:04.683021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f985ef-a3e7-4773-b95c-3cd3f1d5a7c8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.739231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930ade0-b8f1-4d61-ac42-3f51838ecfb3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.744366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:04.775910 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ab2cd-ad72-4cfb-bc31-a6fa5f500219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.776928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ab2cd-ad72-4cfb-bc31-a6fa5f500219 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.777547 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ab2cd-ad72-4cfb-bc31-a6fa5f500219 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.778133 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ab2cd-ad72-4cfb-bc31-a6fa5f500219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.779092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ab2cd-ad72-4cfb-bc31-a6fa5f500219 None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=a4b53c9f4ff842e08ec41f53147d0431) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:04.779593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ab2cd-ad72-4cfb-bc31-a6fa5f500219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.781293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ab2cd-ad72-4cfb-bc31-a6fa5f500219 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.794386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 278/538] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/role_assignments?user.id=a4b53c9f4ff842e08ec41f53147d0431&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:04.795778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ab2cd-ad72-4cfb-bc31-a6fa5f500219 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.812842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f985ef-a3e7-4773-b95c-3cd3f1d5a7c8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.845244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c450fb0-76e8-41f1-b9de-e1ba2412828b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.845820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c450fb0-76e8-41f1-b9de-e1ba2412828b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.846095 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c450fb0-76e8-41f1-b9de-e1ba2412828b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.846360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c450fb0-76e8-41f1-b9de-e1ba2412828b None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.847056 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c450fb0-76e8-41f1-b9de-e1ba2412828b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:04.847354 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c450fb0-76e8-41f1-b9de-e1ba2412828b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.848959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c450fb0-76e8-41f1-b9de-e1ba2412828b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.881499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 261/539] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:04.887989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c450fb0-76e8-41f1-b9de-e1ba2412828b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.987636 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:05.004002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ab2cd-ad72-4cfb-bc31-a6fa5f500219 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:05.007763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c450fb0-76e8-41f1-b9de-e1ba2412828b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:05.019552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:05.048337 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33af798-165c-430a-8f85-49191fa03930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:05.049063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33af798-165c-430a-8f85-49191fa03930 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.049441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33af798-165c-430a-8f85-49191fa03930 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.049823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33af798-165c-430a-8f85-49191fa03930 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.050663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33af798-165c-430a-8f85-49191fa03930 None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=5e18461c8cc4497eae7a83d9ad1448f6) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:05.051046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33af798-165c-430a-8f85-49191fa03930 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:05.052868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33af798-165c-430a-8f85-49191fa03930 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:05.057195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afde58-1ddb-4515-87e1-e1a5489ea0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:05.057990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afde58-1ddb-4515-87e1-e1a5489ea0cb None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.058485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afde58-1ddb-4515-87e1-e1a5489ea0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.059003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afde58-1ddb-4515-87e1-e1a5489ea0cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.060032 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 262/540] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/role_assignments?user.id=5e18461c8cc4497eae7a83d9ad1448f6&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:05.060240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afde58-1ddb-4515-87e1-e1a5489ea0cb None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=f94bfed415d24a9cb596b1fa92d5df97) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:05.060706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afde58-1ddb-4515-87e1-e1a5489ea0cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:05.062413 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afde58-1ddb-4515-87e1-e1a5489ea0cb None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:05.072938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 279/541] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/role_assignments?user.id=f94bfed415d24a9cb596b1fa92d5df97&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:05.172899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f75433-16d9-44d6-a41b-d4544cae0843 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.173183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f75433-16d9-44d6-a41b-d4544cae0843 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.173469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f75433-16d9-44d6-a41b-d4544cae0843 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.175252 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 263/542] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:05.181206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401f8fb-10de-4685-b749-a4ec7995eb40 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.181494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401f8fb-10de-4685-b749-a4ec7995eb40 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.181771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401f8fb-10de-4685-b749-a4ec7995eb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.201897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d401f8fb-10de-4685-b749-a4ec7995eb40 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:05.203139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d401f8fb-10de-4685-b749-a4ec7995eb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:05.281133 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 280/543] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:05.556899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f75433-16d9-44d6-a41b-d4544cae0843 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:05.689369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f75433-16d9-44d6-a41b-d4544cae0843 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:05.726978 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fc9a8-9d8e-4174-8739-79a7cf11c0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:05.727618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fc9a8-9d8e-4174-8739-79a7cf11c0de None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.727899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fc9a8-9d8e-4174-8739-79a7cf11c0de None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.728172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fc9a8-9d8e-4174-8739-79a7cf11c0de None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fc9a8-9d8e-4174-8739-79a7cf11c0de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-918fc9a8-9d8e-4174-8739-79a7cf11c0de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:05.743942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:05.744837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:05.746106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 264/544] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:05.751236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401f8fb-10de-4685-b749-a4ec7995eb40 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:05.843229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:05.854975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401f8fb-10de-4685-b749-a4ec7995eb40 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:05.890466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3061d1-aa47-4756-b9e9-306864c324be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:05.891076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3061d1-aa47-4756-b9e9-306864c324be None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.891351 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3061d1-aa47-4756-b9e9-306864c324be None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.891617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3061d1-aa47-4756-b9e9-306864c324be None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.892282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3061d1-aa47-4756-b9e9-306864c324be None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:05.892579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3061d1-aa47-4756-b9e9-306864c324be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:05.893990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3061d1-aa47-4756-b9e9-306864c324be None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:05.903339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 281/545] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:05.908961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fc9a8-9d8e-4174-8739-79a7cf11c0de None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:05.945542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:06.034794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fc9a8-9d8e-4174-8739-79a7cf11c0de None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:06.072856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486d5aa-fc6e-47e5-9439-bae290844c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:06.073498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486d5aa-fc6e-47e5-9439-bae290844c90 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.073776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486d5aa-fc6e-47e5-9439-bae290844c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.074073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486d5aa-fc6e-47e5-9439-bae290844c90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.091969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486d5aa-fc6e-47e5-9439-bae290844c90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:06.093757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486d5aa-fc6e-47e5-9439-bae290844c90 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c486d5aa-fc6e-47e5-9439-bae290844c90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:06.108843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:06.109781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:06.112865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:06.112865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:06.112865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:06.112865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 265/546] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:06.117165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3061d1-aa47-4756-b9e9-306864c324be None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:06.322404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3061d1-aa47-4756-b9e9-306864c324be None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:06.382562 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60144919-bf2c-4671-8578-70ceb605d0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:06.383556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60144919-bf2c-4671-8578-70ceb605d0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.384059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60144919-bf2c-4671-8578-70ceb605d0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.384516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60144919-bf2c-4671-8578-70ceb605d0c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.385379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60144919-bf2c-4671-8578-70ceb605d0c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:06.385866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60144919-bf2c-4671-8578-70ceb605d0c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:06.387583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60144919-bf2c-4671-8578-70ceb605d0c1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:06.405002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 282/547] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:06.413147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486d5aa-fc6e-47e5-9439-bae290844c90 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:06.424738 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6c4fc-50ed-4173-bebd-1764c4373257 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.425372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6c4fc-50ed-4173-bebd-1764c4373257 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.426035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6c4fc-50ed-4173-bebd-1764c4373257 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.427626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 283/548] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:06.437337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca49ca-6d3d-48f1-9db3-aeadfd2f1c92 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.437780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca49ca-6d3d-48f1-9db3-aeadfd2f1c92 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.438261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca49ca-6d3d-48f1-9db3-aeadfd2f1c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.462035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-cfca49ca-6d3d-48f1-9db3-aeadfd2f1c92 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:06.463422 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cfca49ca-6d3d-48f1-9db3-aeadfd2f1c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:06.475936 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:06.590885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 284/549] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:06.658863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:06.687281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486d5aa-fc6e-47e5-9439-bae290844c90 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:06.753460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf03ac-ae43-4ca4-8937-f5f9119b235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:06.754118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf03ac-ae43-4ca4-8937-f5f9119b235a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.754398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf03ac-ae43-4ca4-8937-f5f9119b235a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.754665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf03ac-ae43-4ca4-8937-f5f9119b235a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.773746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf03ac-ae43-4ca4-8937-f5f9119b235a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:06.775250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf03ac-ae43-4ca4-8937-f5f9119b235a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-04cf03ac-ae43-4ca4-8937-f5f9119b235a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:06.793262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:06.795480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:06.796861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:06.796861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:06.796861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:06.796861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:06.796861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:06.796861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:06.796861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:06.796861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:06.796861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:06.796861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:06.796861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 266/550] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:06.807033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca49ca-6d3d-48f1-9db3-aeadfd2f1c92 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.012367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca49ca-6d3d-48f1-9db3-aeadfd2f1c92 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.102014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444b6b2-505d-4daa-aca1-de05408e4df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.103011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444b6b2-505d-4daa-aca1-de05408e4df0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.103300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444b6b2-505d-4daa-aca1-de05408e4df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.103575 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444b6b2-505d-4daa-aca1-de05408e4df0 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.104330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444b6b2-505d-4daa-aca1-de05408e4df0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:07.104630 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444b6b2-505d-4daa-aca1-de05408e4df0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.106339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444b6b2-505d-4daa-aca1-de05408e4df0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.137115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 285/551] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 24 12:20:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:07.137359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.145885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf03ac-ae43-4ca4-8937-f5f9119b235a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.152768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444b6b2-505d-4daa-aca1-de05408e4df0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.327338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444b6b2-505d-4daa-aca1-de05408e4df0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.357004 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf03ac-ae43-4ca4-8937-f5f9119b235a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.402398 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530574e4-4d31-4a9b-b8a1-9bcefd79fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.403113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530574e4-4d31-4a9b-b8a1-9bcefd79fa6d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.403403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530574e4-4d31-4a9b-b8a1-9bcefd79fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.403667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530574e4-4d31-4a9b-b8a1-9bcefd79fa6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.415668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04687470-2914-4cc7-a0ca-6c27092ab749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.416677 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04687470-2914-4cc7-a0ca-6c27092ab749 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.417100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04687470-2914-4cc7-a0ca-6c27092ab749 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.417513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04687470-2914-4cc7-a0ca-6c27092ab749 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.437041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.437621 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530574e4-4d31-4a9b-b8a1-9bcefd79fa6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-530574e4-4d31-4a9b-b8a1-9bcefd79fa6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:07.442835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:07.443767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:07.444755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:07.444755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:07.444755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:07.444755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:07.444755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:07.444755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:07.444755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:07.444755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:07.444755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:07.444755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:07.444755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:07.444755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:07.448634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 286/552] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:07.453733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530574e4-4d31-4a9b-b8a1-9bcefd79fa6d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.501360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04687470-2914-4cc7-a0ca-6c27092ab749 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.503543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04687470-2914-4cc7-a0ca-6c27092ab749 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.562288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.609350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530574e4-4d31-4a9b-b8a1-9bcefd79fa6d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.631330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 267/553] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:07.632628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3791d-3b37-4db6-9cc8-b2bf018f6e3c None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.632905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3791d-3b37-4db6-9cc8-b2bf018f6e3c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.633269 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3791d-3b37-4db6-9cc8-b2bf018f6e3c None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.636692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 268/554] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:07.636692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546d8f1-d24f-4840-a46d-8c51567581db None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.636692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546d8f1-d24f-4840-a46d-8c51567581db None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.636692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546d8f1-d24f-4840-a46d-8c51567581db None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.637440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 269/555] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:07.638368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb43a34-d6a6-4889-b114-f89f75ed4d98 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.638647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb43a34-d6a6-4889-b114-f89f75ed4d98 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.638908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb43a34-d6a6-4889-b114-f89f75ed4d98 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.640266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 270/556] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:07.641167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb43a34-d6a6-4889-b114-f89f75ed4d98 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.654709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fd4d6-3f07-4374-9482-7853ce5dd578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.655394 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd4d6-3f07-4374-9482-7853ce5dd578 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.655671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd4d6-3f07-4374-9482-7853ce5dd578 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.655937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd4d6-3f07-4374-9482-7853ce5dd578 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.656598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fd4d6-3f07-4374-9482-7853ce5dd578 None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:07.656928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fd4d6-3f07-4374-9482-7853ce5dd578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.658300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fd4d6-3f07-4374-9482-7853ce5dd578 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.666486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 287/557] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:07.668843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047172d-aabd-4225-9549-553d7ccf35bc None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.669151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047172d-aabd-4225-9549-553d7ccf35bc None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.669458 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047172d-aabd-4225-9549-553d7ccf35bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.686752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6047172d-aabd-4225-9549-553d7ccf35bc None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:07.687920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6047172d-aabd-4225-9549-553d7ccf35bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:07.692178 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.749550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb43a34-d6a6-4889-b114-f89f75ed4d98 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.755478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 288/558] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:07.756825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce26f5b-a852-4bd0-95ba-59b17aff8d2d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.757104 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce26f5b-a852-4bd0-95ba-59b17aff8d2d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.757529 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce26f5b-a852-4bd0-95ba-59b17aff8d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.784350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb7bc7-e556-4f98-bcd3-fc51a5121b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb7bc7-e556-4f98-bcd3-fc51a5121b69 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.785258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-dce26f5b-a852-4bd0-95ba-59b17aff8d2d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:07.785360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb7bc7-e556-4f98-bcd3-fc51a5121b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.785571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb7bc7-e556-4f98-bcd3-fc51a5121b69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.787316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dce26f5b-a852-4bd0-95ba-59b17aff8d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:07.799713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb7bc7-e556-4f98-bcd3-fc51a5121b69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.801517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb7bc7-e556-4f98-bcd3-fc51a5121b69 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-9acb7bc7-e556-4f98-bcd3-fc51a5121b69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:07.822126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:07.823175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:07.824282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:07.824282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:07.824282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:07.826048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 271/559] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:07.827465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474592cc-c351-43a3-b707-eb540f29945b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.827883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474592cc-c351-43a3-b707-eb540f29945b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.828270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474592cc-c351-43a3-b707-eb540f29945b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.841948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.851195 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-474592cc-c351-43a3-b707-eb540f29945b None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:07.852736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-474592cc-c351-43a3-b707-eb540f29945b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:07.867756 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 289/560] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:07.868875 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce26f5b-a852-4bd0-95ba-59b17aff8d2d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.935495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 272/561] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:07.936416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474592cc-c351-43a3-b707-eb540f29945b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.993009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.041268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce26f5b-a852-4bd0-95ba-59b17aff8d2d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.074298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474592cc-c351-43a3-b707-eb540f29945b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.075749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f10c5f-0477-4858-9763-4178363651db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.076495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f10c5f-0477-4858-9763-4178363651db None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.076892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f10c5f-0477-4858-9763-4178363651db None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.077221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f10c5f-0477-4858-9763-4178363651db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.087841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f10c5f-0477-4858-9763-4178363651db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.089799 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f10c5f-0477-4858-9763-4178363651db None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-b5f10c5f-0477-4858-9763-4178363651db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:08.105102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:08.106253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:08.107110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:08.107110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:08.107110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:08.111082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 290/562] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:08.111967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f10c5f-0477-4858-9763-4178363651db None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.114459 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212f485-0415-40a5-a219-6befb2531f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.115364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212f485-0415-40a5-a219-6befb2531f91 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.115895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212f485-0415-40a5-a219-6befb2531f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.116368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212f485-0415-40a5-a219-6befb2531f91 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.117266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212f485-0415-40a5-a219-6befb2531f91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:08.117818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212f485-0415-40a5-a219-6befb2531f91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.119399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212f485-0415-40a5-a219-6befb2531f91 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.132033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 273/563] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:08.133368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212f485-0415-40a5-a219-6befb2531f91 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.144478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.237035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f10c5f-0477-4858-9763-4178363651db None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.238216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.254365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212f485-0415-40a5-a219-6befb2531f91 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.272556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16f94b-7d65-47d1-b3ae-80f15ba688b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.273223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16f94b-7d65-47d1-b3ae-80f15ba688b2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.273498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16f94b-7d65-47d1-b3ae-80f15ba688b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.273767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16f94b-7d65-47d1-b3ae-80f15ba688b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.284361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16f94b-7d65-47d1-b3ae-80f15ba688b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:08.288754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a1dfa-5425-4651-9e15-71edc5aa3f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.289356 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a1dfa-5425-4651-9e15-71edc5aa3f58 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-1e16f94b-7d65-47d1-b3ae-80f15ba688b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:08.289500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:08.290474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:08.291295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:08.291295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:08.291295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:08.291295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:08.291295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:08.291295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:08.291295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:08.291295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:08.291295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:08.291295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:08.291295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:08.291295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:08.291922 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a1dfa-5425-4651-9e15-71edc5aa3f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.291922 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a1dfa-5425-4651-9e15-71edc5aa3f58 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.291922 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a1dfa-5425-4651-9e15-71edc5aa3f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:08.292109 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 291/564] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:08.292176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a1dfa-5425-4651-9e15-71edc5aa3f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.292243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16f94b-7d65-47d1-b3ae-80f15ba688b2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.292309 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a1dfa-5425-4651-9e15-71edc5aa3f58 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.305126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 274/565] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:08.305946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a1dfa-5425-4651-9e15-71edc5aa3f58 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.387613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.398206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16f94b-7d65-47d1-b3ae-80f15ba688b2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.406052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a1dfa-5425-4651-9e15-71edc5aa3f58 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.431912 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780bb30-7722-4ba6-a0be-5841df538b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.432543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780bb30-7722-4ba6-a0be-5841df538b22 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.432853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780bb30-7722-4ba6-a0be-5841df538b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.433125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780bb30-7722-4ba6-a0be-5841df538b22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.439894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780bb30-7722-4ba6-a0be-5841df538b22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.440940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e053d-0e54-43fa-a5d3-cc8a93d55373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.441379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780bb30-7722-4ba6-a0be-5841df538b22 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.441536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e053d-0e54-43fa-a5d3-cc8a93d55373 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.441840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e053d-0e54-43fa-a5d3-cc8a93d55373 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.442110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e053d-0e54-43fa-a5d3-cc8a93d55373 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.452375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e053d-0e54-43fa-a5d3-cc8a93d55373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6780bb30-7722-4ba6-a0be-5841df538b22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:08.452677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:08.453678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:08.454736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:08.454736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:08.454736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:08.454736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:08.454736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:08.454736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:08.454736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:08.454736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:08.456970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 292/566] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b68e053d-0e54-43fa-a5d3-cc8a93d55373 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:08.457070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:08.458272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:08.459647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:08.459647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:08.459647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:08.459647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:08.459647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:08.459647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:08.459647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:08.459647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:08.459647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:08.459647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:08.459647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:08.459647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:08.460497 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780bb30-7722-4ba6-a0be-5841df538b22 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.460589 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 275/567] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:08.460589 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e053d-0e54-43fa-a5d3-cc8a93d55373 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.469475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.560822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e053d-0e54-43fa-a5d3-cc8a93d55373 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.562266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780bb30-7722-4ba6-a0be-5841df538b22 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.596709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ed00d-021d-4701-b62b-994269e3cde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.596966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b22fc-1367-4d32-a4b6-ca8c672928cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.597313 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed00d-021d-4701-b62b-994269e3cde9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.597408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b22fc-1367-4d32-a4b6-ca8c672928cf None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.597588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed00d-021d-4701-b62b-994269e3cde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.597693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b22fc-1367-4d32-a4b6-ca8c672928cf None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.597859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed00d-021d-4701-b62b-994269e3cde9 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.597948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b22fc-1367-4d32-a4b6-ca8c672928cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.598524 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ed00d-021d-4701-b62b-994269e3cde9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:08.598826 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ed00d-021d-4701-b62b-994269e3cde9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.600260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ed00d-021d-4701-b62b-994269e3cde9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.608911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b22fc-1367-4d32-a4b6-ca8c672928cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:08.613497 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 276/568] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:08.613497 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ed00d-021d-4701-b62b-994269e3cde9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-a02b22fc-1367-4d32-a4b6-ca8c672928cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:08.615048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:08.616120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:08.616973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 293/569] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:08.621537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b22fc-1367-4d32-a4b6-ca8c672928cf None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.634009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.679213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.713949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ed00d-021d-4701-b62b-994269e3cde9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.721874 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b22fc-1367-4d32-a4b6-ca8c672928cf None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.746794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19154792-1f3b-487a-8715-5a6c96900d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.747416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19154792-1f3b-487a-8715-5a6c96900d69 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.747699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19154792-1f3b-487a-8715-5a6c96900d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.747961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19154792-1f3b-487a-8715-5a6c96900d69 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.755493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19154792-1f3b-487a-8715-5a6c96900d69 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.755618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48836765-fe68-4727-91b2-1e2939187701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.756112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48836765-fe68-4727-91b2-1e2939187701 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.756388 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48836765-fe68-4727-91b2-1e2939187701 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.756674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48836765-fe68-4727-91b2-1e2939187701 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.756988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19154792-1f3b-487a-8715-5a6c96900d69 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.757372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48836765-fe68-4727-91b2-1e2939187701 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:08.757674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48836765-fe68-4727-91b2-1e2939187701 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.759103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48836765-fe68-4727-91b2-1e2939187701 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-19154792-1f3b-487a-8715-5a6c96900d69 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:08.767919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:08.769654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:08.770798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:08.770798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:08.770798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:08.770798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:08.770798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:08.770798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:08.770798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:08.770798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:08.770798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:08.770798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:08.772802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 277/570] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:08.773918 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19154792-1f3b-487a-8715-5a6c96900d69 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.780744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 294/571] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:08.780744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48836765-fe68-4727-91b2-1e2939187701 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.846187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.874006 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19154792-1f3b-487a-8715-5a6c96900d69 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.876724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48836765-fe68-4727-91b2-1e2939187701 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.899990 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.908170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b55fc-5e59-4c27-9ccb-621afd5e22af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.908801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b55fc-5e59-4c27-9ccb-621afd5e22af None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.909080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b55fc-5e59-4c27-9ccb-621afd5e22af None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.909404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b55fc-5e59-4c27-9ccb-621afd5e22af None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.910070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b55fc-5e59-4c27-9ccb-621afd5e22af None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:08.910372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b55fc-5e59-4c27-9ccb-621afd5e22af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.911770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b55fc-5e59-4c27-9ccb-621afd5e22af None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.911931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b391e9a-7a3a-44ab-aaa2-77f75313c648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.912563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b391e9a-7a3a-44ab-aaa2-77f75313c648 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.912872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b391e9a-7a3a-44ab-aaa2-77f75313c648 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.913292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b391e9a-7a3a-44ab-aaa2-77f75313c648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.919985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 278/572] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:08.920806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b55fc-5e59-4c27-9ccb-621afd5e22af None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.928571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b391e9a-7a3a-44ab-aaa2-77f75313c648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.930512 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b391e9a-7a3a-44ab-aaa2-77f75313c648 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3b391e9a-7a3a-44ab-aaa2-77f75313c648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:08.944905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:08.946043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:08.947261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:08.947261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:08.947261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:08.947261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 295/573] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:08.947261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b391e9a-7a3a-44ab-aaa2-77f75313c648 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.025498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b55fc-5e59-4c27-9ccb-621afd5e22af None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.050250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b391e9a-7a3a-44ab-aaa2-77f75313c648 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.059939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755f3a7-2e43-4cd3-87c4-7dfea6cc9dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.060530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755f3a7-2e43-4cd3-87c4-7dfea6cc9dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.060834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755f3a7-2e43-4cd3-87c4-7dfea6cc9dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.061101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755f3a7-2e43-4cd3-87c4-7dfea6cc9dd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.061810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755f3a7-2e43-4cd3-87c4-7dfea6cc9dd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:09.062093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755f3a7-2e43-4cd3-87c4-7dfea6cc9dd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.063473 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755f3a7-2e43-4cd3-87c4-7dfea6cc9dd2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.071759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 279/574] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:09.072396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.072538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755f3a7-2e43-4cd3-87c4-7dfea6cc9dd2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.085257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb6757-7ebe-49a6-a197-1ee4ae8eb9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.085869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb6757-7ebe-49a6-a197-1ee4ae8eb9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.086145 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb6757-7ebe-49a6-a197-1ee4ae8eb9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.086415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb6757-7ebe-49a6-a197-1ee4ae8eb9c1 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.087152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb6757-7ebe-49a6-a197-1ee4ae8eb9c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:09.087455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb6757-7ebe-49a6-a197-1ee4ae8eb9c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.089274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb6757-7ebe-49a6-a197-1ee4ae8eb9c1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.107642 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 296/575] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:09.108540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb6757-7ebe-49a6-a197-1ee4ae8eb9c1 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.127107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.178679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755f3a7-2e43-4cd3-87c4-7dfea6cc9dd2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.209027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb6757-7ebe-49a6-a197-1ee4ae8eb9c1 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.212882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8a0cc-6979-45ad-9b45-b758a7438927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.213823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8a0cc-6979-45ad-9b45-b758a7438927 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.214097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8a0cc-6979-45ad-9b45-b758a7438927 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.214365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8a0cc-6979-45ad-9b45-b758a7438927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.215416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8a0cc-6979-45ad-9b45-b758a7438927 None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=bfec040b09c9415b9e89cd28c67aa5df) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:09.215704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8a0cc-6979-45ad-9b45-b758a7438927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.217200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8a0cc-6979-45ad-9b45-b758a7438927 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.222690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 280/576] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/role_assignments?user.id=bfec040b09c9415b9e89cd28c67aa5df&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:09.223462 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8a0cc-6979-45ad-9b45-b758a7438927 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.246032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f5866-0f37-4959-ab48-81e9cda4c4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.246685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f5866-0f37-4959-ab48-81e9cda4c4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.246961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f5866-0f37-4959-ab48-81e9cda4c4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.247226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f5866-0f37-4959-ab48-81e9cda4c4d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.259441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f5866-0f37-4959-ab48-81e9cda4c4d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.260988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f5866-0f37-4959-ab48-81e9cda4c4d5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-e23f5866-0f37-4959-ab48-81e9cda4c4d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:09.277565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:09.278735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:09.279807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:09.279807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:09.279807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:09.283766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 297/577] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:09.284776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f5866-0f37-4959-ab48-81e9cda4c4d5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.319403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.345352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8a0cc-6979-45ad-9b45-b758a7438927 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.363498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.388270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91588ff4-62ab-441e-b7fd-bf01be6a3713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.388915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91588ff4-62ab-441e-b7fd-bf01be6a3713 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.389196 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91588ff4-62ab-441e-b7fd-bf01be6a3713 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.389468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91588ff4-62ab-441e-b7fd-bf01be6a3713 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.390121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91588ff4-62ab-441e-b7fd-bf01be6a3713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:09.390425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91588ff4-62ab-441e-b7fd-bf01be6a3713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.391842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91588ff4-62ab-441e-b7fd-bf01be6a3713 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.395986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f5866-0f37-4959-ab48-81e9cda4c4d5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.404504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 281/578] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:09.409692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91588ff4-62ab-441e-b7fd-bf01be6a3713 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.430271 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fdd96-aeac-4a86-b896-a11502c2b3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.430898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fdd96-aeac-4a86-b896-a11502c2b3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.431223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fdd96-aeac-4a86-b896-a11502c2b3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.431494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fdd96-aeac-4a86-b896-a11502c2b3c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.443464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fdd96-aeac-4a86-b896-a11502c2b3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.445185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fdd96-aeac-4a86-b896-a11502c2b3c0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-569fdd96-aeac-4a86-b896-a11502c2b3c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:09.460717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:09.461678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:09.462534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:09.462534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:09.462534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:09.466823 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 298/579] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:09.467627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fdd96-aeac-4a86-b896-a11502c2b3c0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.513009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91588ff4-62ab-441e-b7fd-bf01be6a3713 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.549287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95eb3f-2830-4b18-aa08-47b33c99fea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.549928 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95eb3f-2830-4b18-aa08-47b33c99fea4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.550206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95eb3f-2830-4b18-aa08-47b33c99fea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.550477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95eb3f-2830-4b18-aa08-47b33c99fea4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.559896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.561174 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95eb3f-2830-4b18-aa08-47b33c99fea4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.562658 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95eb3f-2830-4b18-aa08-47b33c99fea4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.577179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fdd96-aeac-4a86-b896-a11502c2b3c0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-5d95eb3f-2830-4b18-aa08-47b33c99fea4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:09.577944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:09.579030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:09.580061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:09.580061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:09.580061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:09.580061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 282/580] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:09.580291 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95eb3f-2830-4b18-aa08-47b33c99fea4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.613847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.621901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4834556-f94a-41b0-bdee-b619d032f1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.622511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4834556-f94a-41b0-bdee-b619d032f1ca None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.622783 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4834556-f94a-41b0-bdee-b619d032f1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.623050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4834556-f94a-41b0-bdee-b619d032f1ca None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.623692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4834556-f94a-41b0-bdee-b619d032f1ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:09.623987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4834556-f94a-41b0-bdee-b619d032f1ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.625418 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4834556-f94a-41b0-bdee-b619d032f1ca None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.637683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 299/581] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:09.643024 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4834556-f94a-41b0-bdee-b619d032f1ca None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.693976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95eb3f-2830-4b18-aa08-47b33c99fea4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.731525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fabc3-de43-4b1b-ba99-f9397d1a580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.732200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fabc3-de43-4b1b-ba99-f9397d1a580c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.732514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fabc3-de43-4b1b-ba99-f9397d1a580c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.732788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fabc3-de43-4b1b-ba99-f9397d1a580c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.742410 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404fabc3-de43-4b1b-ba99-f9397d1a580c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.743859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404fabc3-de43-4b1b-ba99-f9397d1a580c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.754671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4834556-f94a-41b0-bdee-b619d032f1ca None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-404fabc3-de43-4b1b-ba99-f9397d1a580c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:09.755449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:09.756347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:09.757755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:09.757755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:09.757755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:09.757755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:09.757755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:09.757755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:09.757755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:09.757755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:09.757755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:20:09.757755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:09.757755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 283/582] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:09.757755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fabc3-de43-4b1b-ba99-f9397d1a580c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.799286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631f009-528c-43a8-bbc3-a367650e3905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.799889 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631f009-528c-43a8-bbc3-a367650e3905 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.800169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631f009-528c-43a8-bbc3-a367650e3905 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.800439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631f009-528c-43a8-bbc3-a367650e3905 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.801121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631f009-528c-43a8-bbc3-a367650e3905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:09.801421 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631f009-528c-43a8-bbc3-a367650e3905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.804480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631f009-528c-43a8-bbc3-a367650e3905 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.820618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 300/583] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:09.820618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631f009-528c-43a8-bbc3-a367650e3905 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.822998 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.867280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.881188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fabc3-de43-4b1b-ba99-f9397d1a580c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.917090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0992c25-a2e1-4524-afb2-8335a8d1da7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.917785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0992c25-a2e1-4524-afb2-8335a8d1da7d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.918635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0992c25-a2e1-4524-afb2-8335a8d1da7d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.918915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0992c25-a2e1-4524-afb2-8335a8d1da7d None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.919569 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0992c25-a2e1-4524-afb2-8335a8d1da7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:09.919874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0992c25-a2e1-4524-afb2-8335a8d1da7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.921299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0992c25-a2e1-4524-afb2-8335a8d1da7d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.932456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631f009-528c-43a8-bbc3-a367650e3905 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.939322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 284/584] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:09.939322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0992c25-a2e1-4524-afb2-8335a8d1da7d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.969295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb290c1-9506-40f9-a443-aa8955da336e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.969924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb290c1-9506-40f9-a443-aa8955da336e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.970206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb290c1-9506-40f9-a443-aa8955da336e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.970477 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb290c1-9506-40f9-a443-aa8955da336e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.985277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb290c1-9506-40f9-a443-aa8955da336e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.987040 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb290c1-9506-40f9-a443-aa8955da336e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6bb290c1-9506-40f9-a443-aa8955da336e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:10.000341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:10.001320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:10.002223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:10.002223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:10.002223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:10.002223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:10.002223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:10.002223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:10.002223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:10.002223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:10.002223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:10.002223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:10.006189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 301/585] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:10.006825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb290c1-9506-40f9-a443-aa8955da336e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.047492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0992c25-a2e1-4524-afb2-8335a8d1da7d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.084944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8f67d-a889-437a-ac4c-dcc7caa1709b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.085544 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8f67d-a889-437a-ac4c-dcc7caa1709b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.085824 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8f67d-a889-437a-ac4c-dcc7caa1709b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.086099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8f67d-a889-437a-ac4c-dcc7caa1709b None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.086824 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8f67d-a889-437a-ac4c-dcc7caa1709b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:10.087125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8f67d-a889-437a-ac4c-dcc7caa1709b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.087427 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.088973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8f67d-a889-437a-ac4c-dcc7caa1709b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.127958 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 285/586] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:10.128837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8f67d-a889-437a-ac4c-dcc7caa1709b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.134740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb290c1-9506-40f9-a443-aa8955da336e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.138655 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.171578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7415c-c5ce-4f97-b084-722b96e9b7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.172202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7415c-c5ce-4f97-b084-722b96e9b7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.172473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7415c-c5ce-4f97-b084-722b96e9b7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.172769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7415c-c5ce-4f97-b084-722b96e9b7b0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.180626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7415c-c5ce-4f97-b084-722b96e9b7b0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.182107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7415c-c5ce-4f97-b084-722b96e9b7b0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-4aa7415c-c5ce-4f97-b084-722b96e9b7b0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:10.195647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:10.196674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:10.197599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:10.197599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:10.197599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:10.197599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:10.197599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:10.197599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:10.197599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:10.197599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:10.197599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:20:10.197599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:10.197599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 302/587] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:10.197599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7415c-c5ce-4f97-b084-722b96e9b7b0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.246716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8f67d-a889-437a-ac4c-dcc7caa1709b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.287117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a9fc5-62cd-4176-9a32-8ff5b00c92f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.287779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a9fc5-62cd-4176-9a32-8ff5b00c92f8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.288122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a9fc5-62cd-4176-9a32-8ff5b00c92f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.288402 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a9fc5-62cd-4176-9a32-8ff5b00c92f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.296073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a9fc5-62cd-4176-9a32-8ff5b00c92f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.297975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a9fc5-62cd-4176-9a32-8ff5b00c92f8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-4f5a9fc5-62cd-4176-9a32-8ff5b00c92f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:10.309397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:10.310277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:10.311147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:10.311147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:10.311147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:10.311147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:10.311147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:10.311147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:10.311147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:10.311147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:10.311147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 286/588] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:10.311619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a9fc5-62cd-4176-9a32-8ff5b00c92f8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.317955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7415c-c5ce-4f97-b084-722b96e9b7b0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.344231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.356316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070a51c-952d-45aa-987d-81cff742110b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.356951 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070a51c-952d-45aa-987d-81cff742110b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.357582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070a51c-952d-45aa-987d-81cff742110b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.357859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070a51c-952d-45aa-987d-81cff742110b None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.358595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1070a51c-952d-45aa-987d-81cff742110b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:10.358896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1070a51c-952d-45aa-987d-81cff742110b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.360476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1070a51c-952d-45aa-987d-81cff742110b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.378506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 303/589] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:10.379305 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070a51c-952d-45aa-987d-81cff742110b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.390049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.426625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a9fc5-62cd-4176-9a32-8ff5b00c92f8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.461954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b8de3-82f7-40d7-adc4-b500bd3d1a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.462613 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b8de3-82f7-40d7-adc4-b500bd3d1a28 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.462885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b8de3-82f7-40d7-adc4-b500bd3d1a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.463153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b8de3-82f7-40d7-adc4-b500bd3d1a28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.474326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b8de3-82f7-40d7-adc4-b500bd3d1a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.475812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b8de3-82f7-40d7-adc4-b500bd3d1a28 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.485272 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070a51c-952d-45aa-987d-81cff742110b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-659b8de3-82f7-40d7-adc4-b500bd3d1a28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:10.489698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:10.490622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:10.491528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:10.491528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:10.491528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:10.491528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 287/590] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:10.491756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b8de3-82f7-40d7-adc4-b500bd3d1a28 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.524083 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a514ae-52be-4850-989b-fd313221808a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.524961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a514ae-52be-4850-989b-fd313221808a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.525481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a514ae-52be-4850-989b-fd313221808a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.525888 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a514ae-52be-4850-989b-fd313221808a None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.526896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a514ae-52be-4850-989b-fd313221808a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:10.527346 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a514ae-52be-4850-989b-fd313221808a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.529341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a514ae-52be-4850-989b-fd313221808a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.552532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 304/591] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:10.553369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a514ae-52be-4850-989b-fd313221808a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.599079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.603860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b8de3-82f7-40d7-adc4-b500bd3d1a28 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.621942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.640842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba10560-5f03-4d22-be3b-661e5ba266c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.641471 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba10560-5f03-4d22-be3b-661e5ba266c4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.641733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba10560-5f03-4d22-be3b-661e5ba266c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.642002 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba10560-5f03-4d22-be3b-661e5ba266c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.642699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba10560-5f03-4d22-be3b-661e5ba266c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:10.643006 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba10560-5f03-4d22-be3b-661e5ba266c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.644475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba10560-5f03-4d22-be3b-661e5ba266c4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.657742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 288/592] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:10.658395 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba10560-5f03-4d22-be3b-661e5ba266c4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.670306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a514ae-52be-4850-989b-fd313221808a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.707992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0531b-698d-4bb4-b8fe-46eac2f2b857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.708624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0531b-698d-4bb4-b8fe-46eac2f2b857 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.708933 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0531b-698d-4bb4-b8fe-46eac2f2b857 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.709204 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0531b-698d-4bb4-b8fe-46eac2f2b857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.721600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0531b-698d-4bb4-b8fe-46eac2f2b857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.723179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0531b-698d-4bb4-b8fe-46eac2f2b857 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-d5a0531b-698d-4bb4-b8fe-46eac2f2b857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:10.739208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:10.740191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:10.741744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:10.741744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:10.741744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:10.741744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 305/593] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:10.741744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0531b-698d-4bb4-b8fe-46eac2f2b857 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.768369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba10560-5f03-4d22-be3b-661e5ba266c4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.808641 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbc38d-4286-4e5b-a7ed-9af04101905e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.809224 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbc38d-4286-4e5b-a7ed-9af04101905e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.809556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbc38d-4286-4e5b-a7ed-9af04101905e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.809831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbc38d-4286-4e5b-a7ed-9af04101905e None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.810485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbc38d-4286-4e5b-a7ed-9af04101905e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:10.810782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbc38d-4286-4e5b-a7ed-9af04101905e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.812199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbc38d-4286-4e5b-a7ed-9af04101905e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.823940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 289/594] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:10.824884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbc38d-4286-4e5b-a7ed-9af04101905e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.850515 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.861500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0531b-698d-4bb4-b8fe-46eac2f2b857 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.868193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.898971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5ca18-fbbd-4673-acd2-3f6444fb7fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.899755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5ca18-fbbd-4673-acd2-3f6444fb7fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.900136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5ca18-fbbd-4673-acd2-3f6444fb7fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.900514 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5ca18-fbbd-4673-acd2-3f6444fb7fa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.911716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5ca18-fbbd-4673-acd2-3f6444fb7fa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.913236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5ca18-fbbd-4673-acd2-3f6444fb7fa1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-c6d5ca18-fbbd-4673-acd2-3f6444fb7fa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:10.927653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:10.928592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:10.929767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:10.929767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:10.929767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:10.931136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 306/595] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:10.932051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5ca18-fbbd-4673-acd2-3f6444fb7fa1 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.939963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbc38d-4286-4e5b-a7ed-9af04101905e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.982722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250c52c-063b-42e7-bb5c-7a61a4cad352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.983483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250c52c-063b-42e7-bb5c-7a61a4cad352 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.983826 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250c52c-063b-42e7-bb5c-7a61a4cad352 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.984249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250c52c-063b-42e7-bb5c-7a61a4cad352 None admin] PATH_INFO: `/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/e65026e01df242e7be11ed24214a4efa/roles/248565f1fe7849c5b05a207eb214cbb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.001017 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250c52c-063b-42e7-bb5c-7a61a4cad352 None admin] RBAC: Authorizing `identity:create_grant(project_id=70a5f9cf0499405d82f5f5a47c2877eb, user_id=e65026e01df242e7be11ed24214a4efa, role_id=248565f1fe7849c5b05a207eb214cbb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.002793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250c52c-063b-42e7-bb5c-7a61a4cad352 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.017515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 290/596] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:10 2022] PUT /identity/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/e65026e01df242e7be11ed24214a4efa/roles/248565f1fe7849c5b05a207eb214cbb7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:11.018575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250c52c-063b-42e7-bb5c-7a61a4cad352 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.069058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5ca18-fbbd-4673-acd2-3f6444fb7fa1 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.121692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21263f37-ec0b-4eb2-b7f1-e702e63833da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.122316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21263f37-ec0b-4eb2-b7f1-e702e63833da None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.122597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21263f37-ec0b-4eb2-b7f1-e702e63833da None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.122926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21263f37-ec0b-4eb2-b7f1-e702e63833da None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.124170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21263f37-ec0b-4eb2-b7f1-e702e63833da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:11.124478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21263f37-ec0b-4eb2-b7f1-e702e63833da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.126318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21263f37-ec0b-4eb2-b7f1-e702e63833da None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.138034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.144937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 307/597] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:11.144937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21263f37-ec0b-4eb2-b7f1-e702e63833da None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.148294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.160676 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250c52c-063b-42e7-bb5c-7a61a4cad352 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.204468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88d632-0602-456f-ac7d-2d8a180fb1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.205172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88d632-0602-456f-ac7d-2d8a180fb1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.205449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88d632-0602-456f-ac7d-2d8a180fb1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.205714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88d632-0602-456f-ac7d-2d8a180fb1b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.212605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88d632-0602-456f-ac7d-2d8a180fb1b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.214444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88d632-0602-456f-ac7d-2d8a180fb1b8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-ba88d632-0602-456f-ac7d-2d8a180fb1b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:11.225635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:11.226716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:11.229912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:11.229912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:11.229912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:11.229912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:11.229912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:11.229912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:11.229912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:11.229912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:11.229912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 291/598] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:11.229912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc04fc-4378-430a-bc40-3c3f4c9fe010 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.229912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc04fc-4378-430a-bc40-3c3f4c9fe010 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.231060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc04fc-4378-430a-bc40-3c3f4c9fe010 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.231060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 292/599] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:11.231060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc04fc-4378-430a-bc40-3c3f4c9fe010 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.264919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21263f37-ec0b-4eb2-b7f1-e702e63833da None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.299002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6cfbd-0837-4375-9373-fc93438c0dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.299609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6cfbd-0837-4375-9373-fc93438c0dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.299891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6cfbd-0837-4375-9373-fc93438c0dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.300157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6cfbd-0837-4375-9373-fc93438c0dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.300828 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6cfbd-0837-4375-9373-fc93438c0dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:11.301130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6cfbd-0837-4375-9373-fc93438c0dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.302502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6cfbd-0837-4375-9373-fc93438c0dd4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.314874 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 308/600] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:11.315721 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6cfbd-0837-4375-9373-fc93438c0dd4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.341997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc04fc-4378-430a-bc40-3c3f4c9fe010 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.376268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18cc7e-313b-4712-9a87-10634ed5a421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.376953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18cc7e-313b-4712-9a87-10634ed5a421 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.377262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18cc7e-313b-4712-9a87-10634ed5a421 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.377498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18cc7e-313b-4712-9a87-10634ed5a421 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.381574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.384399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18cc7e-313b-4712-9a87-10634ed5a421 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.385916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18cc7e-313b-4712-9a87-10634ed5a421 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-aa18cc7e-313b-4712-9a87-10634ed5a421 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:11.397192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:11.398088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:11.398932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:11.398932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:11.398932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:11.398932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:11.398932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:11.398932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:11.398932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:11.398932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:11.398932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 293/601] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:11.400012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031c0c0-4630-4bc5-99e4-2e07a7164825 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.400012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031c0c0-4630-4bc5-99e4-2e07a7164825 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.400012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031c0c0-4630-4bc5-99e4-2e07a7164825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.417853 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f031c0c0-4630-4bc5-99e4-2e07a7164825 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:11.419047 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f031c0c0-4630-4bc5-99e4-2e07a7164825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:11.421118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.427174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6cfbd-0837-4375-9373-fc93438c0dd4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.478013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660edac5-275b-49d2-8036-8825bbddfa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.480122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660edac5-275b-49d2-8036-8825bbddfa55 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.480122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660edac5-275b-49d2-8036-8825bbddfa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.480122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660edac5-275b-49d2-8036-8825bbddfa55 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.480122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660edac5-275b-49d2-8036-8825bbddfa55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:11.480122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660edac5-275b-49d2-8036-8825bbddfa55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.482449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660edac5-275b-49d2-8036-8825bbddfa55 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.495990 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 294/602] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:11.497085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031c0c0-4630-4bc5-99e4-2e07a7164825 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.507250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 309/603] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:11.508883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660edac5-275b-49d2-8036-8825bbddfa55 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.609314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031c0c0-4630-4bc5-99e4-2e07a7164825 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.649316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660edac5-275b-49d2-8036-8825bbddfa55 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.650440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf4f92-b680-4cf4-93d0-825f274b86ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.651303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf4f92-b680-4cf4-93d0-825f274b86ad None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.651828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf4f92-b680-4cf4-93d0-825f274b86ad None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.652374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf4f92-b680-4cf4-93d0-825f274b86ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.660417 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf4f92-b680-4cf4-93d0-825f274b86ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.662412 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf4f92-b680-4cf4-93d0-825f274b86ad None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.664621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f6cf4f92-b680-4cf4-93d0-825f274b86ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:11.677011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:11.677930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:11.678800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:11.678800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:11.678800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:11.678800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:11.678800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:11.678800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:11.678800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:11.678800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:11.679667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.681355 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 295/604] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:11.682523 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf4f92-b680-4cf4-93d0-825f274b86ad None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.691950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728838a0-fac1-4ef4-894e-651e3b2c6335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.692562 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728838a0-fac1-4ef4-894e-651e3b2c6335 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.692890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728838a0-fac1-4ef4-894e-651e3b2c6335 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.693157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728838a0-fac1-4ef4-894e-651e3b2c6335 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.694356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728838a0-fac1-4ef4-894e-651e3b2c6335 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:11.694657 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728838a0-fac1-4ef4-894e-651e3b2c6335 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.696504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728838a0-fac1-4ef4-894e-651e3b2c6335 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.716721 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 310/605] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:11.718636 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728838a0-fac1-4ef4-894e-651e3b2c6335 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.796719 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf4f92-b680-4cf4-93d0-825f274b86ad None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.833327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728838a0-fac1-4ef4-894e-651e3b2c6335 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.834150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89467187-b838-42a2-a22b-93527739ce1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.834811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467187-b838-42a2-a22b-93527739ce1e None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.835102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467187-b838-42a2-a22b-93527739ce1e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.835370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467187-b838-42a2-a22b-93527739ce1e None admin] PATH_INFO: `/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/f94bfed415d24a9cb596b1fa92d5df97/roles/248565f1fe7849c5b05a207eb214cbb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.854847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89467187-b838-42a2-a22b-93527739ce1e None admin] RBAC: Authorizing `identity:create_grant(project_id=70a5f9cf0499405d82f5f5a47c2877eb, user_id=f94bfed415d24a9cb596b1fa92d5df97, role_id=248565f1fe7849c5b05a207eb214cbb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.856557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89467187-b838-42a2-a22b-93527739ce1e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.869227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6efd1a-e292-4231-b7b1-a090d3c36e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.869761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6efd1a-e292-4231-b7b1-a090d3c36e07 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.870032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6efd1a-e292-4231-b7b1-a090d3c36e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.870296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6efd1a-e292-4231-b7b1-a090d3c36e07 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.871369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6efd1a-e292-4231-b7b1-a090d3c36e07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:11.871728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 296/606] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:11 2022] PUT /identity/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/f94bfed415d24a9cb596b1fa92d5df97/roles/248565f1fe7849c5b05a207eb214cbb7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:11.872553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89467187-b838-42a2-a22b-93527739ce1e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.877040 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6efd1a-e292-4231-b7b1-a090d3c36e07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.878603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6efd1a-e292-4231-b7b1-a090d3c36e07 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.900751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 311/607] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:11.902273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6efd1a-e292-4231-b7b1-a090d3c36e07 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.934705 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.973614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:12.012637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89467187-b838-42a2-a22b-93527739ce1e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.053696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6efd1a-e292-4231-b7b1-a090d3c36e07 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.060525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4a86e-05b6-4637-bdd7-738f53d669f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.061252 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4a86e-05b6-4637-bdd7-738f53d669f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.061526 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4a86e-05b6-4637-bdd7-738f53d669f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.062290 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4a86e-05b6-4637-bdd7-738f53d669f5 None admin] PATH_INFO: `/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/a4b53c9f4ff842e08ec41f53147d0431/roles/248565f1fe7849c5b05a207eb214cbb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.083609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4a86e-05b6-4637-bdd7-738f53d669f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=70a5f9cf0499405d82f5f5a47c2877eb, user_id=a4b53c9f4ff842e08ec41f53147d0431, role_id=248565f1fe7849c5b05a207eb214cbb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:12.085976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c4a86e-05b6-4637-bdd7-738f53d669f5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:12.097339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126ba40-a1c3-4366-ac59-368957b11bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.098053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126ba40-a1c3-4366-ac59-368957b11bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.098258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126ba40-a1c3-4366-ac59-368957b11bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.098524 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126ba40-a1c3-4366-ac59-368957b11bb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.109279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 297/608] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:11 2022] PUT /identity/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/a4b53c9f4ff842e08ec41f53147d0431/roles/248565f1fe7849c5b05a207eb214cbb7 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:12.110410 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4a86e-05b6-4637-bdd7-738f53d669f5 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:12.111582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126ba40-a1c3-4366-ac59-368957b11bb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-1126ba40-a1c3-4366-ac59-368957b11bb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:12.116667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:12.117870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:12.119015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:12.119015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:12.119015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:12.119015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:12.119015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:12.119015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:12.119015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:12.119015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:12.119015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:12.119015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:12.119015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:12.119015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:12.124943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 312/609] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:12.128446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126ba40-a1c3-4366-ac59-368957b11bb2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:12.317736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:12.322350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4a86e-05b6-4637-bdd7-738f53d669f5 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.333817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126ba40-a1c3-4366-ac59-368957b11bb2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.336691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:12.365634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4c560-a6e0-411d-bd34-34e43da0c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.366307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4c560-a6e0-411d-bd34-34e43da0c91e None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.366583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4c560-a6e0-411d-bd34-34e43da0c91e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.366851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4c560-a6e0-411d-bd34-34e43da0c91e None admin] PATH_INFO: `/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/5e18461c8cc4497eae7a83d9ad1448f6/roles/248565f1fe7849c5b05a207eb214cbb7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.373684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795632da-fe3f-41ef-9d01-3bf04bb0d08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.374425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795632da-fe3f-41ef-9d01-3bf04bb0d08b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.374802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795632da-fe3f-41ef-9d01-3bf04bb0d08b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.375219 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795632da-fe3f-41ef-9d01-3bf04bb0d08b None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.375935 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795632da-fe3f-41ef-9d01-3bf04bb0d08b None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:12.376339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795632da-fe3f-41ef-9d01-3bf04bb0d08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:12.378657 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795632da-fe3f-41ef-9d01-3bf04bb0d08b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:12.384685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4c560-a6e0-411d-bd34-34e43da0c91e None admin] RBAC: Authorizing `identity:create_grant(project_id=70a5f9cf0499405d82f5f5a47c2877eb, user_id=5e18461c8cc4497eae7a83d9ad1448f6, role_id=248565f1fe7849c5b05a207eb214cbb7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:12.386334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4c560-a6e0-411d-bd34-34e43da0c91e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:12.392446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 313/610] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:12.394896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795632da-fe3f-41ef-9d01-3bf04bb0d08b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:12.411522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 298/611] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:12 2022] PUT /identity/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/5e18461c8cc4497eae7a83d9ad1448f6/roles/248565f1fe7849c5b05a207eb214cbb7 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:12.521137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795632da-fe3f-41ef-9d01-3bf04bb0d08b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.559311 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4127b8-7a38-4ec5-8394-23d32ec80a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.559966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4127b8-7a38-4ec5-8394-23d32ec80a86 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.560243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4127b8-7a38-4ec5-8394-23d32ec80a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.560520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4127b8-7a38-4ec5-8394-23d32ec80a86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.572750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4127b8-7a38-4ec5-8394-23d32ec80a86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:12.574305 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4127b8-7a38-4ec5-8394-23d32ec80a86 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-2d4127b8-7a38-4ec5-8394-23d32ec80a86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:12.590224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:12.591387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:12.592294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:12.592294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:12.592294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:12.592294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 314/612] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:12.597444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4c560-a6e0-411d-bd34-34e43da0c91e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:12.704090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4c560-a6e0-411d-bd34-34e43da0c91e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.739520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175e5a9-c422-49a0-990b-96df79e5187d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.740127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175e5a9-c422-49a0-990b-96df79e5187d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.740402 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175e5a9-c422-49a0-990b-96df79e5187d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.740688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175e5a9-c422-49a0-990b-96df79e5187d None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.741338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175e5a9-c422-49a0-990b-96df79e5187d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:12.741637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175e5a9-c422-49a0-990b-96df79e5187d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:12.743034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175e5a9-c422-49a0-990b-96df79e5187d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:12.760307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 299/613] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:12.761912 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4127b8-7a38-4ec5-8394-23d32ec80a86 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:12.783407 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:12.894955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4127b8-7a38-4ec5-8394-23d32ec80a86 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.941209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c6ef3-66f3-470c-81ab-6a3e620803c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.941846 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c6ef3-66f3-470c-81ab-6a3e620803c4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.942124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c6ef3-66f3-470c-81ab-6a3e620803c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.942399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c6ef3-66f3-470c-81ab-6a3e620803c4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.955294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c6ef3-66f3-470c-81ab-6a3e620803c4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:12.956687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c6ef3-66f3-470c-81ab-6a3e620803c4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-ea8c6ef3-66f3-470c-81ab-6a3e620803c4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:12.977425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:12.978461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:12.979387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:12.979387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:12.979387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:12.979387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:12.979387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:12.979387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:12.979387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:12.979387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:12.979387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:12.979387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:12.979387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 315/614] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:12.984921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175e5a9-c422-49a0-990b-96df79e5187d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:12.997062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:13.092617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175e5a9-c422-49a0-990b-96df79e5187d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.127324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c1da9-9824-49a7-b9d1-abcf0c02e972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.127930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c1da9-9824-49a7-b9d1-abcf0c02e972 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.128207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c1da9-9824-49a7-b9d1-abcf0c02e972 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.128479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c1da9-9824-49a7-b9d1-abcf0c02e972 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.129241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c1da9-9824-49a7-b9d1-abcf0c02e972 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:13.129545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c1da9-9824-49a7-b9d1-abcf0c02e972 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.131062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c1da9-9824-49a7-b9d1-abcf0c02e972 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.138375 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e35fdf-324e-469d-affe-3b2814c8af26 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.138890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e35fdf-324e-469d-affe-3b2814c8af26 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.138890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e35fdf-324e-469d-affe-3b2814c8af26 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.140239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 316/615] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:13.145367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e925239-9af8-419e-8b53-c44ea551712e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.145645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e925239-9af8-419e-8b53-c44ea551712e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.145914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e925239-9af8-419e-8b53-c44ea551712e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.157453 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 300/616] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 24 12:20:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:13.159690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c1da9-9824-49a7-b9d1-abcf0c02e972 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.193678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6e925239-9af8-419e-8b53-c44ea551712e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:13.194887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6e925239-9af8-419e-8b53-c44ea551712e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:13.229206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:13.263746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c1da9-9824-49a7-b9d1-abcf0c02e972 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.300194 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a944f-3a33-44a9-ae9c-e3f4e8a491f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.300855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a944f-3a33-44a9-ae9c-e3f4e8a491f6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.301132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a944f-3a33-44a9-ae9c-e3f4e8a491f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.301398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a944f-3a33-44a9-ae9c-e3f4e8a491f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.312873 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a944f-3a33-44a9-ae9c-e3f4e8a491f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.314581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a944f-3a33-44a9-ae9c-e3f4e8a491f6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-394a944f-3a33-44a9-ae9c-e3f4e8a491f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:13.330474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:13.331461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:13.332321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:13.332321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:13.332321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:13.336543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 301/617] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:13.337928 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a944f-3a33-44a9-ae9c-e3f4e8a491f6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.355677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 317/618] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:13.445495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a944f-3a33-44a9-ae9c-e3f4e8a491f6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.463756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:13.483040 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3609423-9081-467e-8627-aac714cf2a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.483566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3609423-9081-467e-8627-aac714cf2a7f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.483842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3609423-9081-467e-8627-aac714cf2a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.484179 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3609423-9081-467e-8627-aac714cf2a7f None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.484862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3609423-9081-467e-8627-aac714cf2a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:13.485212 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3609423-9081-467e-8627-aac714cf2a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.486623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3609423-9081-467e-8627-aac714cf2a7f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.503857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 302/619] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:13.509769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e925239-9af8-419e-8b53-c44ea551712e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.524762 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:13.648885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e925239-9af8-419e-8b53-c44ea551712e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.691289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3609423-9081-467e-8627-aac714cf2a7f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.700692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe087ced-2dab-413e-8534-040e80e6b78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.701434 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe087ced-2dab-413e-8534-040e80e6b78a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.701711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe087ced-2dab-413e-8534-040e80e6b78a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.701978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe087ced-2dab-413e-8534-040e80e6b78a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.714782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe087ced-2dab-413e-8534-040e80e6b78a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.716321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe087ced-2dab-413e-8534-040e80e6b78a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-fe087ced-2dab-413e-8534-040e80e6b78a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:13.733367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:13.734459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:13.735308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:13.735308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:13.735308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:13.735308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:13.735308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:13.735308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:13.735308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:13.735308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:13.739223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 318/620] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:13.741626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe087ced-2dab-413e-8534-040e80e6b78a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.800502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3609423-9081-467e-8627-aac714cf2a7f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.847977 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deff4d1-1c99-4353-828c-8d82146c4d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.848621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deff4d1-1c99-4353-828c-8d82146c4d3a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.848919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deff4d1-1c99-4353-828c-8d82146c4d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.849189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deff4d1-1c99-4353-828c-8d82146c4d3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.881322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deff4d1-1c99-4353-828c-8d82146c4d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:13.882546 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-2deff4d1-1c99-4353-828c-8d82146c4d3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:13.886015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:13.887058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:13.887862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:13.887862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:13.887862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:13.887862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:13.887862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:13.887862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:13.887862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:13.887862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:13.887862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:13.887862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:13.887862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:13.887862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:13.888494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 303/621] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:13.897647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deff4d1-1c99-4353-828c-8d82146c4d3a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.928726 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe087ced-2dab-413e-8534-040e80e6b78a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.967984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d5cd4-e6a0-4522-b67e-54b563618a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.968627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d5cd4-e6a0-4522-b67e-54b563618a61 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.968934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d5cd4-e6a0-4522-b67e-54b563618a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.969206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d5cd4-e6a0-4522-b67e-54b563618a61 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.969914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d5cd4-e6a0-4522-b67e-54b563618a61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:13.970219 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d5cd4-e6a0-4522-b67e-54b563618a61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.971676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d5cd4-e6a0-4522-b67e-54b563618a61 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.988781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 319/622] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:13.993373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d5cd4-e6a0-4522-b67e-54b563618a61 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:14.026614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:14.105834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d5cd4-e6a0-4522-b67e-54b563618a61 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:14.136031 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:14.151788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32982b0c-51f6-4c2b-9607-2766f7f578a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:14.153572 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32982b0c-51f6-4c2b-9607-2766f7f578a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.154702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32982b0c-51f6-4c2b-9607-2766f7f578a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.155332 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32982b0c-51f6-4c2b-9607-2766f7f578a6 None admin] PATH_INFO: `/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/bfec040b09c9415b9e89cd28c67aa5df/roles/248565f1fe7849c5b05a207eb214cbb7` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.171679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deff4d1-1c99-4353-828c-8d82146c4d3a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:14.187276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32982b0c-51f6-4c2b-9607-2766f7f578a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=70a5f9cf0499405d82f5f5a47c2877eb, user_id=bfec040b09c9415b9e89cd28c67aa5df, role_id=248565f1fe7849c5b05a207eb214cbb7)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:14.190717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32982b0c-51f6-4c2b-9607-2766f7f578a6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:14.213213 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 320/623] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:13 2022] PUT /identity/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/bfec040b09c9415b9e89cd28c67aa5df/roles/248565f1fe7849c5b05a207eb214cbb7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:20:14.231894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5fcd3-ec48-4dae-a1f6-de7f68cd0510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:14.233002 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5fcd3-ec48-4dae-a1f6-de7f68cd0510 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.233489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5fcd3-ec48-4dae-a1f6-de7f68cd0510 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.234021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5fcd3-ec48-4dae-a1f6-de7f68cd0510 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.234911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5fcd3-ec48-4dae-a1f6-de7f68cd0510 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:14.235456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5fcd3-ec48-4dae-a1f6-de7f68cd0510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:14.237461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5fcd3-ec48-4dae-a1f6-de7f68cd0510 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:14.248023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 304/624] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:14.254426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32982b0c-51f6-4c2b-9607-2766f7f578a6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:14.396614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32982b0c-51f6-4c2b-9607-2766f7f578a6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:14.428505 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574bdc8-e2bb-41fb-a68e-0e3a4e369772 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.429030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574bdc8-e2bb-41fb-a68e-0e3a4e369772 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.429441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574bdc8-e2bb-41fb-a68e-0e3a4e369772 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.430988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 305/625] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:14.440970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa118d97-aa4f-42e4-a9c4-7583643e48f2 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.441251 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa118d97-aa4f-42e4-a9c4-7583643e48f2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.441527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa118d97-aa4f-42e4-a9c4-7583643e48f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.456348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262a46f-4adc-4076-9c78-a1053c1a1580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:14.457310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262a46f-4adc-4076-9c78-a1053c1a1580 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.457722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262a46f-4adc-4076-9c78-a1053c1a1580 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.458188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262a46f-4adc-4076-9c78-a1053c1a1580 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.466615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-aa118d97-aa4f-42e4-a9c4-7583643e48f2 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:14.467862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-aa118d97-aa4f-42e4-a9c4-7583643e48f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:14.471101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262a46f-4adc-4076-9c78-a1053c1a1580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:14.472785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262a46f-4adc-4076-9c78-a1053c1a1580 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-f262a46f-4adc-4076-9c78-a1053c1a1580 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:14.491683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:14.492922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:14.494001 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:14.494001 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:14.494001 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:14.496163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 321/626] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:14.498127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9973f-5525-4e57-818c-03b113616375 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.498743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9973f-5525-4e57-818c-03b113616375 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.499148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9973f-5525-4e57-818c-03b113616375 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.500997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 322/627] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:14.505969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9973f-5525-4e57-818c-03b113616375 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:14.544852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:14.562604 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 306/628] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:14.563913 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d67d09-e1b5-4311-872e-ac9b198a3cfa None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.564197 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d67d09-e1b5-4311-872e-ac9b198a3cfa None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.564464 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d67d09-e1b5-4311-872e-ac9b198a3cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.584910 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a2d67d09-e1b5-4311-872e-ac9b198a3cfa None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:14.586211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a2d67d09-e1b5-4311-872e-ac9b198a3cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:14.613393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:14.638486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9973f-5525-4e57-818c-03b113616375 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:14.662710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 307/629] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:14.684294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8d74e-6fc3-4838-a4c7-a088de03dbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:14.684953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8d74e-6fc3-4838-a4c7-a088de03dbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.685231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8d74e-6fc3-4838-a4c7-a088de03dbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.685498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8d74e-6fc3-4838-a4c7-a088de03dbb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.686142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8d74e-6fc3-4838-a4c7-a088de03dbb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:14.686436 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8d74e-6fc3-4838-a4c7-a088de03dbb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:14.687864 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8d74e-6fc3-4838-a4c7-a088de03dbb2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:14.709918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 323/630] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:14.715926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d67d09-e1b5-4311-872e-ac9b198a3cfa None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:14.834328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d67d09-e1b5-4311-872e-ac9b198a3cfa None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:14.876808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b285c8-6480-404c-9201-f243ca2dc7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:14.877627 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b285c8-6480-404c-9201-f243ca2dc7c9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.878003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b285c8-6480-404c-9201-f243ca2dc7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.878366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b285c8-6480-404c-9201-f243ca2dc7c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.886200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b285c8-6480-404c-9201-f243ca2dc7c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:14.888011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b285c8-6480-404c-9201-f243ca2dc7c9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b2b285c8-6480-404c-9201-f243ca2dc7c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:14.900327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:14.901565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:14.902682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:14.902682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:14.902682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:14.902682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:14.902682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:14.902682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:14.902682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:14.902682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:14.902682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:14.902682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:14.904420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 308/631] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 12:20:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:14.913760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8d74e-6fc3-4838-a4c7-a088de03dbb2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:14.918864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b285c8-6480-404c-9201-f243ca2dc7c9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:14.949391 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:15.030785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:15.042913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8d74e-6fc3-4838-a4c7-a088de03dbb2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:15.045932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b285c8-6480-404c-9201-f243ca2dc7c9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:15.079847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179149b1-07ca-4be2-9f84-ac7434aaa316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:15.080200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf674b-2669-4846-80f7-4d372b3ed5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:15.080447 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179149b1-07ca-4be2-9f84-ac7434aaa316 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.080744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179149b1-07ca-4be2-9f84-ac7434aaa316 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.080844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf674b-2669-4846-80f7-4d372b3ed5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.081013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179149b1-07ca-4be2-9f84-ac7434aaa316 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.081107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf674b-2669-4846-80f7-4d372b3ed5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.081359 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf674b-2669-4846-80f7-4d372b3ed5c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.081736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179149b1-07ca-4be2-9f84-ac7434aaa316 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:15.082210 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179149b1-07ca-4be2-9f84-ac7434aaa316 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:15.083879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179149b1-07ca-4be2-9f84-ac7434aaa316 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:15.091985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf674b-2669-4846-80f7-4d372b3ed5c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-22cf674b-2669-4846-80f7-4d372b3ed5c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:15.097118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:15.098215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 309/632] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:15.101813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a179b-7538-4638-ad85-0377e354b375 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.102565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a179b-7538-4638-ad85-0377e354b375 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.102565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a179b-7538-4638-ad85-0377e354b375 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.102565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 310/633] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:15.102565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a179b-7538-4638-ad85-0377e354b375 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:15.115818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 324/634] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 12:20:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:15.115818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179149b1-07ca-4be2-9f84-ac7434aaa316 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:15.194215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:15.219298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a179b-7538-4638-ad85-0377e354b375 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:15.227409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179149b1-07ca-4be2-9f84-ac7434aaa316 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:15.254326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c23668-9221-4156-9c87-245c9613cc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:15.254946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c23668-9221-4156-9c87-245c9613cc78 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.255221 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c23668-9221-4156-9c87-245c9613cc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.255492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c23668-9221-4156-9c87-245c9613cc78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.266129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c23668-9221-4156-9c87-245c9613cc78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:15.267933 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1da48f-2845-4033-bdc5-b893d342b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:15.268724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1da48f-2845-4033-bdc5-b893d342b01b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.269105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1da48f-2845-4033-bdc5-b893d342b01b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.269484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1da48f-2845-4033-bdc5-b893d342b01b None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.270249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1da48f-2845-4033-bdc5-b893d342b01b None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:15.270676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1da48f-2845-4033-bdc5-b893d342b01b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d8c23668-9221-4156-9c87-245c9613cc78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:15.271215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:15.272284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:15.273384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:15.273384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:15.273384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:15.273384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:15.273384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:15.273384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:15.273384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:15.273384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:15.273384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:15.273384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:15.273384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:15.273384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:15.274827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1da48f-2845-4033-bdc5-b893d342b01b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:15.277405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 311/635] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:15.279082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb380f-067d-45ce-a516-0b35f4943ee8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.279434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb380f-067d-45ce-a516-0b35f4943ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.279707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb380f-067d-45ce-a516-0b35f4943ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.289201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 325/636] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:15.289201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1da48f-2845-4033-bdc5-b893d342b01b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:15.300866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:15.303310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7fbb380f-067d-45ce-a516-0b35f4943ee8 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:15.304565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7fbb380f-067d-45ce-a516-0b35f4943ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:15.388673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 312/637] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:15.388673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb380f-067d-45ce-a516-0b35f4943ee8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:15.398070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1da48f-2845-4033-bdc5-b893d342b01b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:15.436578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935961ce-d27f-403c-b015-ae7db4e065c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:15.437305 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935961ce-d27f-403c-b015-ae7db4e065c4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.437677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935961ce-d27f-403c-b015-ae7db4e065c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.438064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935961ce-d27f-403c-b015-ae7db4e065c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.470814 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:15.504224 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb380f-067d-45ce-a516-0b35f4943ee8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:15.518173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935961ce-d27f-403c-b015-ae7db4e065c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:15.519636 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935961ce-d27f-403c-b015-ae7db4e065c4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:15.532051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:15.540917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563336f-0472-43e2-a654-3ef09ff8c7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:15.541539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563336f-0472-43e2-a654-3ef09ff8c7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.541821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563336f-0472-43e2-a654-3ef09ff8c7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.542092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563336f-0472-43e2-a654-3ef09ff8c7e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.542880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563336f-0472-43e2-a654-3ef09ff8c7e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:15.543186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563336f-0472-43e2-a654-3ef09ff8c7e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:15.545120 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563336f-0472-43e2-a654-3ef09ff8c7e2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:15.558880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 313/638] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:15.559886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563336f-0472-43e2-a654-3ef09ff8c7e2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:15.627073 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 326/639] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:15.628362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935961ce-d27f-403c-b015-ae7db4e065c4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:15.664618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563336f-0472-43e2-a654-3ef09ff8c7e2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:15.696800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:15.702389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463caa2c-7b70-4586-bae9-1d4fa89dd71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:15.703488 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463caa2c-7b70-4586-bae9-1d4fa89dd71a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.703488 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463caa2c-7b70-4586-bae9-1d4fa89dd71a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.703662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463caa2c-7b70-4586-bae9-1d4fa89dd71a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.716101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463caa2c-7b70-4586-bae9-1d4fa89dd71a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:15.717718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463caa2c-7b70-4586-bae9-1d4fa89dd71a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-463caa2c-7b70-4586-bae9-1d4fa89dd71a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:15.733192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:15.734162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:15.739821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:15.739821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:15.739821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:15.739989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935961ce-d27f-403c-b015-ae7db4e065c4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:15.740062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 314/640] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:15.740062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463caa2c-7b70-4586-bae9-1d4fa89dd71a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:15.751279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:15.777303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1106b-95a1-4b00-b9f1-febf46459c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:15.777940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1106b-95a1-4b00-b9f1-febf46459c3c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.778217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1106b-95a1-4b00-b9f1-febf46459c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.778489 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1106b-95a1-4b00-b9f1-febf46459c3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.788775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1106b-95a1-4b00-b9f1-febf46459c3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:15.790221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1106b-95a1-4b00-b9f1-febf46459c3c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-aed1106b-95a1-4b00-b9f1-febf46459c3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:15.805044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:15.805956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:15.806817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:15.806817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:15.806817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:15.806817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 327/641] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:15.807041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1106b-95a1-4b00-b9f1-febf46459c3c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:15.859015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463caa2c-7b70-4586-bae9-1d4fa89dd71a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:15.905070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529099ef-148d-41bd-bc07-969a7d25cde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:15.906067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529099ef-148d-41bd-bc07-969a7d25cde5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.906683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529099ef-148d-41bd-bc07-969a7d25cde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.907048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529099ef-148d-41bd-bc07-969a7d25cde5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.919332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529099ef-148d-41bd-bc07-969a7d25cde5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:15.920783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529099ef-148d-41bd-bc07-969a7d25cde5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:15.931055 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1106b-95a1-4b00-b9f1-febf46459c3c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-529099ef-148d-41bd-bc07-969a7d25cde5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:15.936237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:15.937622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:15.938597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:15.938597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:15.938597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:15.941002 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 315/642] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:15.942215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529099ef-148d-41bd-bc07-969a7d25cde5 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:15.970137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363fa6e-0f40-433e-a740-62ac8177e123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:15.970774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363fa6e-0f40-433e-a740-62ac8177e123 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.971049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363fa6e-0f40-433e-a740-62ac8177e123 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.971322 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363fa6e-0f40-433e-a740-62ac8177e123 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.985981 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363fa6e-0f40-433e-a740-62ac8177e123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:15.991066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6363fa6e-0f40-433e-a740-62ac8177e123 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:15.991188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:15.992028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 328/643] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:15.994229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363fa6e-0f40-433e-a740-62ac8177e123 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.024239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:16.062506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529099ef-148d-41bd-bc07-969a7d25cde5 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.097303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a060bf-16f4-4b9f-91bc-049d6013f71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.097911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a060bf-16f4-4b9f-91bc-049d6013f71c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.098198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a060bf-16f4-4b9f-91bc-049d6013f71c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.098521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a060bf-16f4-4b9f-91bc-049d6013f71c None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.099189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a060bf-16f4-4b9f-91bc-049d6013f71c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:16.099489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a060bf-16f4-4b9f-91bc-049d6013f71c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:16.100904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a060bf-16f4-4b9f-91bc-049d6013f71c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:16.111994 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363fa6e-0f40-433e-a740-62ac8177e123 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.114001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 316/644] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:16.114730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a060bf-16f4-4b9f-91bc-049d6013f71c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.152432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422035ed-6553-4b98-81b8-c38601dcae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.153066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422035ed-6553-4b98-81b8-c38601dcae51 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.153344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422035ed-6553-4b98-81b8-c38601dcae51 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.153611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422035ed-6553-4b98-81b8-c38601dcae51 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.154252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422035ed-6553-4b98-81b8-c38601dcae51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:16.154555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422035ed-6553-4b98-81b8-c38601dcae51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:16.156002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422035ed-6553-4b98-81b8-c38601dcae51 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:16.173139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 329/645] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:16.173139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422035ed-6553-4b98-81b8-c38601dcae51 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.211860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:16.218902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a060bf-16f4-4b9f-91bc-049d6013f71c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.257400 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9ecb7-3331-42d8-afe8-2e4ccf93e501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.258752 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9ecb7-3331-42d8-afe8-2e4ccf93e501 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.259032 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9ecb7-3331-42d8-afe8-2e4ccf93e501 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.259299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9ecb7-3331-42d8-afe8-2e4ccf93e501 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.259947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9ecb7-3331-42d8-afe8-2e4ccf93e501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:16.260244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9ecb7-3331-42d8-afe8-2e4ccf93e501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:16.261925 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9ecb7-3331-42d8-afe8-2e4ccf93e501 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:16.263981 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:16.273653 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 317/646] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:16.279165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9ecb7-3331-42d8-afe8-2e4ccf93e501 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.285346 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422035ed-6553-4b98-81b8-c38601dcae51 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.321835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b1183-8bc1-4637-bc70-7d4337c4fe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.322532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b1183-8bc1-4637-bc70-7d4337c4fe91 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.322815 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b1183-8bc1-4637-bc70-7d4337c4fe91 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.323087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b1183-8bc1-4637-bc70-7d4337c4fe91 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.323765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b1183-8bc1-4637-bc70-7d4337c4fe91 None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:16.324099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b1183-8bc1-4637-bc70-7d4337c4fe91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:16.325620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b1183-8bc1-4637-bc70-7d4337c4fe91 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:16.333787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 330/647] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:16.334644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b1183-8bc1-4637-bc70-7d4337c4fe91 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.394801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9ecb7-3331-42d8-afe8-2e4ccf93e501 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.432390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cccb64-f531-4272-8af5-5eff0da0ba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.432964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cccb64-f531-4272-8af5-5eff0da0ba0a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.433353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cccb64-f531-4272-8af5-5eff0da0ba0a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.433532 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cccb64-f531-4272-8af5-5eff0da0ba0a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.440475 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b1183-8bc1-4637-bc70-7d4337c4fe91 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.441621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cccb64-f531-4272-8af5-5eff0da0ba0a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:16.443082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cccb64-f531-4272-8af5-5eff0da0ba0a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-96cccb64-f531-4272-8af5-5eff0da0ba0a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:16.454921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:16.457375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:16.458284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:16.458284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:16.458284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:16.458284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:16.458284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:16.458284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:16.458284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:16.458284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:16.458284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:16.458284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:16.458284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 318/648] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:16.458284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cccb64-f531-4272-8af5-5eff0da0ba0a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.479564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:16.485422 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb807794-2107-4487-b3b6-1aa12609f880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.486059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb807794-2107-4487-b3b6-1aa12609f880 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.486662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb807794-2107-4487-b3b6-1aa12609f880 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.486662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb807794-2107-4487-b3b6-1aa12609f880 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.495556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb807794-2107-4487-b3b6-1aa12609f880 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:16.497142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb807794-2107-4487-b3b6-1aa12609f880 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-bb807794-2107-4487-b3b6-1aa12609f880 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:16.507939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:16.509046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:16.510366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:16.510366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:16.510366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:16.510366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:16.510366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:16.510366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:16.510366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:16.510366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:16.510366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:20:16.510366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:16.510366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 331/649] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:16.510366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb807794-2107-4487-b3b6-1aa12609f880 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.516977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:16.569859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cccb64-f531-4272-8af5-5eff0da0ba0a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.603334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d613cbf-ec79-48ff-a0de-c4ec71b9200d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.603943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d613cbf-ec79-48ff-a0de-c4ec71b9200d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.604217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d613cbf-ec79-48ff-a0de-c4ec71b9200d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.604517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d613cbf-ec79-48ff-a0de-c4ec71b9200d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.611710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d613cbf-ec79-48ff-a0de-c4ec71b9200d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:16.613205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d613cbf-ec79-48ff-a0de-c4ec71b9200d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:16.615034 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb807794-2107-4487-b3b6-1aa12609f880 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-5d613cbf-ec79-48ff-a0de-c4ec71b9200d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:16.624398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:16.625470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:16.626343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:16.626343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:16.626343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:16.626343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:16.626343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:16.626343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:16.626343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:16.626343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:16.626343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 319/650] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:16.626343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d613cbf-ec79-48ff-a0de-c4ec71b9200d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.652847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee55639-3938-4cfb-a34b-0a7ff4a639a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.653483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee55639-3938-4cfb-a34b-0a7ff4a639a3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.653761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee55639-3938-4cfb-a34b-0a7ff4a639a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.654035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee55639-3938-4cfb-a34b-0a7ff4a639a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.664890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee55639-3938-4cfb-a34b-0a7ff4a639a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:16.666473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee55639-3938-4cfb-a34b-0a7ff4a639a3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-cee55639-3938-4cfb-a34b-0a7ff4a639a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:16.680574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:16.681716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:16.682647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:16.682647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:16.682647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:16.682647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 332/651] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:16.683287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a9af9-932c-475d-80d9-65a3ec043ea4 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.683564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a9af9-932c-475d-80d9-65a3ec043ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.683836 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a9af9-932c-475d-80d9-65a3ec043ea4 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.685092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 333/652] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:16.685768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a9af9-932c-475d-80d9-65a3ec043ea4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.714261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:16.739435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d613cbf-ec79-48ff-a0de-c4ec71b9200d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.748574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:16.774144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a20627-93a7-41d2-a818-65a8c85fc6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.774759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a20627-93a7-41d2-a818-65a8c85fc6ee None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.775035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a20627-93a7-41d2-a818-65a8c85fc6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.775305 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a20627-93a7-41d2-a818-65a8c85fc6ee None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.776043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a20627-93a7-41d2-a818-65a8c85fc6ee None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:16.776348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a20627-93a7-41d2-a818-65a8c85fc6ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:16.777998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a20627-93a7-41d2-a818-65a8c85fc6ee None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:16.794211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a9af9-932c-475d-80d9-65a3ec043ea4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.796282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 320/653] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:16.797896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a20627-93a7-41d2-a818-65a8c85fc6ee None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.828107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695b864-c927-41e3-b63f-860b793da6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.828736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695b864-c927-41e3-b63f-860b793da6ab None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.829017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695b864-c927-41e3-b63f-860b793da6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.829302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695b864-c927-41e3-b63f-860b793da6ab None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.830022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695b864-c927-41e3-b63f-860b793da6ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:16.830322 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695b864-c927-41e3-b63f-860b793da6ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:16.831774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695b864-c927-41e3-b63f-860b793da6ab None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:16.849349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 334/654] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:16.850287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695b864-c927-41e3-b63f-860b793da6ab None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.905423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a20627-93a7-41d2-a818-65a8c85fc6ee None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.940542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4f1d5-c683-4103-b667-3c375a5265f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.941173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4f1d5-c683-4103-b667-3c375a5265f0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.941444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4f1d5-c683-4103-b667-3c375a5265f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.941715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4f1d5-c683-4103-b667-3c375a5265f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.942423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4f1d5-c683-4103-b667-3c375a5265f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:16.942724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4f1d5-c683-4103-b667-3c375a5265f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:16.944168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4f1d5-c683-4103-b667-3c375a5265f0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:16.945783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:16.952939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695b864-c927-41e3-b63f-860b793da6ab None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.956682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 321/655] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:16.961745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d43999-d09c-4094-b053-0d657af48720 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.961745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d43999-d09c-4094-b053-0d657af48720 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.961745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d43999-d09c-4094-b053-0d657af48720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.967605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:16.978770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-97d43999-d09c-4094-b053-0d657af48720 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:16.980303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-97d43999-d09c-4094-b053-0d657af48720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:16.993124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e2901-82e9-48bc-ba15-b9ede8900783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.993731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e2901-82e9-48bc-ba15-b9ede8900783 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.994005 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e2901-82e9-48bc-ba15-b9ede8900783 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.994272 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e2901-82e9-48bc-ba15-b9ede8900783 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.994914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e2901-82e9-48bc-ba15-b9ede8900783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:16.995209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e2901-82e9-48bc-ba15-b9ede8900783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:16.996608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e2901-82e9-48bc-ba15-b9ede8900783 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.009605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 335/656] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:17.010555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e2901-82e9-48bc-ba15-b9ede8900783 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.050301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 322/657] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:17.051308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d43999-d09c-4094-b053-0d657af48720 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.119576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e2901-82e9-48bc-ba15-b9ede8900783 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.152035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d43999-d09c-4094-b053-0d657af48720 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.154104 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168cf97-517f-41c4-999b-966355c8c123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.154875 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168cf97-517f-41c4-999b-966355c8c123 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.155153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168cf97-517f-41c4-999b-966355c8c123 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.155419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168cf97-517f-41c4-999b-966355c8c123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.178856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.190988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dcfe0-c051-4f10-b417-dee2fb4a69f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.191649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dcfe0-c051-4f10-b417-dee2fb4a69f9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.192030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dcfe0-c051-4f10-b417-dee2fb4a69f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.192412 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dcfe0-c051-4f10-b417-dee2fb4a69f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.201362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.229764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168cf97-517f-41c4-999b-966355c8c123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.231693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168cf97-517f-41c4-999b-966355c8c123 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.269448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dcfe0-c051-4f10-b417-dee2fb4a69f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.271103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dcfe0-c051-4f10-b417-dee2fb4a69f9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.348151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 336/658] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:17.349646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168cf97-517f-41c4-999b-966355c8c123 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.377161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 323/659] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:17.378395 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dcfe0-c051-4f10-b417-dee2fb4a69f9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.404224 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.428406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.454250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168cf97-517f-41c4-999b-966355c8c123 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.483014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dcfe0-c051-4f10-b417-dee2fb4a69f9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.488807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed908315-648d-4b54-bade-6fa16b32fc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.489410 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed908315-648d-4b54-bade-6fa16b32fc1f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.489684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed908315-648d-4b54-bade-6fa16b32fc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.489944 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed908315-648d-4b54-bade-6fa16b32fc1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.490659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed908315-648d-4b54-bade-6fa16b32fc1f None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=e65026e01df242e7be11ed24214a4efa) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:17.490966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed908315-648d-4b54-bade-6fa16b32fc1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.492428 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed908315-648d-4b54-bade-6fa16b32fc1f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.498423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 337/660] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/role_assignments?user.id=e65026e01df242e7be11ed24214a4efa&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:17.499204 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed908315-648d-4b54-bade-6fa16b32fc1f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.520395 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce370a-49ef-4bb9-9518-1d4eb35b1a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.521265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce370a-49ef-4bb9-9518-1d4eb35b1a9a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.521547 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce370a-49ef-4bb9-9518-1d4eb35b1a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.521820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce370a-49ef-4bb9-9518-1d4eb35b1a9a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.530542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce370a-49ef-4bb9-9518-1d4eb35b1a9a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.532046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce370a-49ef-4bb9-9518-1d4eb35b1a9a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-09ce370a-49ef-4bb9-9518-1d4eb35b1a9a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:17.545535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:17.546835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:17.548069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:17.548069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:17.548069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:17.548069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:17.548069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:17.548069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:17.548069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:17.548069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:17.548069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:20:17.548069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:17.551898 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 324/661] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:17.552944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce370a-49ef-4bb9-9518-1d4eb35b1a9a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.626729 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed908315-648d-4b54-bade-6fa16b32fc1f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.651738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.670460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce370a-49ef-4bb9-9518-1d4eb35b1a9a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.672241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cdbd4-99eb-4b72-b53f-49db8e66608f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.672241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cdbd4-99eb-4b72-b53f-49db8e66608f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.672462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cdbd4-99eb-4b72-b53f-49db8e66608f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.672604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cdbd4-99eb-4b72-b53f-49db8e66608f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.682568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.687270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0cdbd4-99eb-4b72-b53f-49db8e66608f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-aa0cdbd4-99eb-4b72-b53f-49db8e66608f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:17.692710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:17.693840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:17.694839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 338/662] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:17.695592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cdbd4-99eb-4b72-b53f-49db8e66608f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.721186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64df292-ab3a-4825-8574-38ad48ba9482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.721748 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64df292-ab3a-4825-8574-38ad48ba9482 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.722734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64df292-ab3a-4825-8574-38ad48ba9482 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.722734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64df292-ab3a-4825-8574-38ad48ba9482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.734489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64df292-ab3a-4825-8574-38ad48ba9482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.736011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64df292-ab3a-4825-8574-38ad48ba9482 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f64df292-ab3a-4825-8574-38ad48ba9482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:17.753001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:17.754013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:17.754987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:17.754987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:17.754987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:17.754987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 325/663] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:17.754987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64df292-ab3a-4825-8574-38ad48ba9482 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.808409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cdbd4-99eb-4b72-b53f-49db8e66608f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.848568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d8730-1996-4681-ad8c-127828f0d3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.849415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d8730-1996-4681-ad8c-127828f0d3bf None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.849723 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d8730-1996-4681-ad8c-127828f0d3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.849985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d8730-1996-4681-ad8c-127828f0d3bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.861014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d8730-1996-4681-ad8c-127828f0d3bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.862478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d8730-1996-4681-ad8c-127828f0d3bf None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.867098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64df292-ab3a-4825-8574-38ad48ba9482 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-fd9d8730-1996-4681-ad8c-127828f0d3bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:17.877922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:17.878905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:17.879755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:17.879755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:17.879755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:17.879755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 339/664] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:17.880014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d8730-1996-4681-ad8c-127828f0d3bf None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.910930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40353dd-4d68-43a3-ba90-49ed4cf2b321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.911647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40353dd-4d68-43a3-ba90-49ed4cf2b321 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.911925 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40353dd-4d68-43a3-ba90-49ed4cf2b321 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.912199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40353dd-4d68-43a3-ba90-49ed4cf2b321 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.912984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40353dd-4d68-43a3-ba90-49ed4cf2b321 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:17.913292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40353dd-4d68-43a3-ba90-49ed4cf2b321 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.914754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40353dd-4d68-43a3-ba90-49ed4cf2b321 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.918365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.931302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.939798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 326/665] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:17.939798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40353dd-4d68-43a3-ba90-49ed4cf2b321 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.003673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d8730-1996-4681-ad8c-127828f0d3bf None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.040991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb3d2a-92d5-46ed-a462-9cdb62baf86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.041787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb3d2a-92d5-46ed-a462-9cdb62baf86e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.041980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb3d2a-92d5-46ed-a462-9cdb62baf86e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.042255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb3d2a-92d5-46ed-a462-9cdb62baf86e None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.043139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb3d2a-92d5-46ed-a462-9cdb62baf86e None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:18.043444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb3d2a-92d5-46ed-a462-9cdb62baf86e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.044922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb3d2a-92d5-46ed-a462-9cdb62baf86e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.058941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 340/666] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:18.058941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb3d2a-92d5-46ed-a462-9cdb62baf86e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.065255 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40353dd-4d68-43a3-ba90-49ed4cf2b321 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.102061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb5df6-bf92-42d1-9b23-b00c61c69cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.102679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb5df6-bf92-42d1-9b23-b00c61c69cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.102953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb5df6-bf92-42d1-9b23-b00c61c69cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.103223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb5df6-bf92-42d1-9b23-b00c61c69cd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.103870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb5df6-bf92-42d1-9b23-b00c61c69cd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:18.104169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb5df6-bf92-42d1-9b23-b00c61c69cd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.105595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb5df6-bf92-42d1-9b23-b00c61c69cd7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.117166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 327/667] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:18.117971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb5df6-bf92-42d1-9b23-b00c61c69cd7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.173566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.183386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb3d2a-92d5-46ed-a462-9cdb62baf86e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.197268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.221475 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd5f01-5b0d-47cc-a15c-7f21e93bd0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.221995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd5f01-5b0d-47cc-a15c-7f21e93bd0fd None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.222273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd5f01-5b0d-47cc-a15c-7f21e93bd0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.222547 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd5f01-5b0d-47cc-a15c-7f21e93bd0fd None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.223185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd5f01-5b0d-47cc-a15c-7f21e93bd0fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:18.223490 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd5f01-5b0d-47cc-a15c-7f21e93bd0fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.224754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb5df6-bf92-42d1-9b23-b00c61c69cd7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.224920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd5f01-5b0d-47cc-a15c-7f21e93bd0fd None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.237244 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 341/668] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:18.238328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd5f01-5b0d-47cc-a15c-7f21e93bd0fd None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.267169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96745a88-525c-41cf-875a-2e69d0795762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.268011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96745a88-525c-41cf-875a-2e69d0795762 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.268449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96745a88-525c-41cf-875a-2e69d0795762 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.268883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96745a88-525c-41cf-875a-2e69d0795762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.342054 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96745a88-525c-41cf-875a-2e69d0795762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.343529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96745a88-525c-41cf-875a-2e69d0795762 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.343789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd5f01-5b0d-47cc-a15c-7f21e93bd0fd None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.379198 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e79ab-2857-4e0e-ac67-07a8f13bb20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.379834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e79ab-2857-4e0e-ac67-07a8f13bb20b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.380079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e79ab-2857-4e0e-ac67-07a8f13bb20b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.380344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e79ab-2857-4e0e-ac67-07a8f13bb20b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.390551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e79ab-2857-4e0e-ac67-07a8f13bb20b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.392020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e79ab-2857-4e0e-ac67-07a8f13bb20b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.402512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-c25e79ab-2857-4e0e-ac67-07a8f13bb20b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:18.408468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:18.409419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:18.410277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:18.410277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:18.410277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:18.414366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 342/669] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:18.415155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e79ab-2857-4e0e-ac67-07a8f13bb20b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.443293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.453859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 328/670] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:18.454593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96745a88-525c-41cf-875a-2e69d0795762 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.532308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e79ab-2857-4e0e-ac67-07a8f13bb20b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.560167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96745a88-525c-41cf-875a-2e69d0795762 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.569234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa2754-92f3-4439-8a9b-72e13649e903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.569966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa2754-92f3-4439-8a9b-72e13649e903 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.570574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa2754-92f3-4439-8a9b-72e13649e903 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.570574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa2754-92f3-4439-8a9b-72e13649e903 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.578648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa2754-92f3-4439-8a9b-72e13649e903 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.580390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa2754-92f3-4439-8a9b-72e13649e903 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-88aa2754-92f3-4439-8a9b-72e13649e903 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:18.595247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:18.596189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:18.597140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:18.597140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:18.597140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:18.597140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:18.597140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:18.597140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:18.597140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:18.597140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:18.597569 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f729ef-94ad-4c71-9f6f-98774ae57ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.597666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 343/671] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:18.597666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa2754-92f3-4439-8a9b-72e13649e903 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.600683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f729ef-94ad-4c71-9f6f-98774ae57ddd None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.602075 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f729ef-94ad-4c71-9f6f-98774ae57ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.602476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f729ef-94ad-4c71-9f6f-98774ae57ddd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.611303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f729ef-94ad-4c71-9f6f-98774ae57ddd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.612929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f729ef-94ad-4c71-9f6f-98774ae57ddd None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-97f729ef-94ad-4c71-9f6f-98774ae57ddd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:18.625728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:18.626959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:18.632848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:18.632848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:18.632848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:18.632848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:18.632848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:18.632848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:18.632848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:18.632848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:18.632848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 329/672] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:18.632848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f729ef-94ad-4c71-9f6f-98774ae57ddd None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.648920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.691113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.708025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa2754-92f3-4439-8a9b-72e13649e903 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.743000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f729ef-94ad-4c71-9f6f-98774ae57ddd None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.744468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29909a-4ddb-4266-97d0-829f97214072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.745103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29909a-4ddb-4266-97d0-829f97214072 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.745379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29909a-4ddb-4266-97d0-829f97214072 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.747217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29909a-4ddb-4266-97d0-829f97214072 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.747217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29909a-4ddb-4266-97d0-829f97214072 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:18.747217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29909a-4ddb-4266-97d0-829f97214072 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.748134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29909a-4ddb-4266-97d0-829f97214072 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.760575 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 344/673] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:18.761259 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29909a-4ddb-4266-97d0-829f97214072 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.784701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c228c48-cbd6-4fcb-a905-b80611a58089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.785269 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c228c48-cbd6-4fcb-a905-b80611a58089 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.785544 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c228c48-cbd6-4fcb-a905-b80611a58089 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.785811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c228c48-cbd6-4fcb-a905-b80611a58089 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.796193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c228c48-cbd6-4fcb-a905-b80611a58089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.797680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c228c48-cbd6-4fcb-a905-b80611a58089 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-4c228c48-cbd6-4fcb-a905-b80611a58089 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:18.813320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:18.814284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:18.815161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:18.815161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:18.815161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:18.815161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 330/674] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:18.815161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c228c48-cbd6-4fcb-a905-b80611a58089 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.866244 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29909a-4ddb-4266-97d0-829f97214072 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.883803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.910581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aeb3c4-5443-460d-bd57-d0687e3dcbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.911186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aeb3c4-5443-460d-bd57-d0687e3dcbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.911458 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aeb3c4-5443-460d-bd57-d0687e3dcbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aeb3c4-5443-460d-bd57-d0687e3dcbe9 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.912931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aeb3c4-5443-460d-bd57-d0687e3dcbe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:18.913146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aeb3c4-5443-460d-bd57-d0687e3dcbe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.914556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aeb3c4-5443-460d-bd57-d0687e3dcbe9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.923106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c228c48-cbd6-4fcb-a905-b80611a58089 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.931555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 345/675] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:18.931555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aeb3c4-5443-460d-bd57-d0687e3dcbe9 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.936321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.959988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2029db1-ed79-441f-9687-5614bc702b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.960594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2029db1-ed79-441f-9687-5614bc702b1b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.960910 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2029db1-ed79-441f-9687-5614bc702b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.961181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2029db1-ed79-441f-9687-5614bc702b1b None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.961877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2029db1-ed79-441f-9687-5614bc702b1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:18.962185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2029db1-ed79-441f-9687-5614bc702b1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.963994 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2029db1-ed79-441f-9687-5614bc702b1b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.984603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 331/676] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:18.984603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2029db1-ed79-441f-9687-5614bc702b1b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.049628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aeb3c4-5443-460d-bd57-d0687e3dcbe9 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.085844 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f712e3-88bb-468a-b5a0-61fc993a58e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.085844 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f712e3-88bb-468a-b5a0-61fc993a58e3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.088100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f712e3-88bb-468a-b5a0-61fc993a58e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.088849 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f712e3-88bb-468a-b5a0-61fc993a58e3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.097880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2029db1-ed79-441f-9687-5614bc702b1b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.101185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f712e3-88bb-468a-b5a0-61fc993a58e3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.102678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f712e3-88bb-468a-b5a0-61fc993a58e3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-62f712e3-88bb-468a-b5a0-61fc993a58e3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:19.125461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:19.126673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:19.128062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:19.128062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:19.128062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:19.128062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:19.128062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:19.128062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:19.128062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:19.128062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:19.128062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:19.128062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:19.128062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 346/677] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:19.133027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f712e3-88bb-468a-b5a0-61fc993a58e3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.140520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.156352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07608886-c54f-42cd-9024-258601fa4a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.157207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07608886-c54f-42cd-9024-258601fa4a35 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.157577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07608886-c54f-42cd-9024-258601fa4a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.157940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07608886-c54f-42cd-9024-258601fa4a35 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.158718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07608886-c54f-42cd-9024-258601fa4a35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:19.159325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07608886-c54f-42cd-9024-258601fa4a35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.160578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07608886-c54f-42cd-9024-258601fa4a35 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.175421 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 332/678] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:19.183170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07608886-c54f-42cd-9024-258601fa4a35 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.208914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.254403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f712e3-88bb-468a-b5a0-61fc993a58e3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.297339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a961a-29ec-4ecc-a2ba-121c2ffdfc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.297953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a961a-29ec-4ecc-a2ba-121c2ffdfc53 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.298226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a961a-29ec-4ecc-a2ba-121c2ffdfc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.298489 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a961a-29ec-4ecc-a2ba-121c2ffdfc53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.299231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a961a-29ec-4ecc-a2ba-121c2ffdfc53 None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=a4b53c9f4ff842e08ec41f53147d0431) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:19.299536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a961a-29ec-4ecc-a2ba-121c2ffdfc53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.301074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a961a-29ec-4ecc-a2ba-121c2ffdfc53 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.308184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 347/679] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/role_assignments?user.id=a4b53c9f4ff842e08ec41f53147d0431&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:19.309222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a961a-29ec-4ecc-a2ba-121c2ffdfc53 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.350801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07608886-c54f-42cd-9024-258601fa4a35 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.405715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b003d9-375f-4a40-a48c-79bd851e7472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.407874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b003d9-375f-4a40-a48c-79bd851e7472 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.408461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b003d9-375f-4a40-a48c-79bd851e7472 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.409908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b003d9-375f-4a40-a48c-79bd851e7472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.411539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b003d9-375f-4a40-a48c-79bd851e7472 None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=f94bfed415d24a9cb596b1fa92d5df97) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:19.412930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b003d9-375f-4a40-a48c-79bd851e7472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.418113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b003d9-375f-4a40-a48c-79bd851e7472 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.436700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 333/680] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/role_assignments?user.id=f94bfed415d24a9cb596b1fa92d5df97&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:19.436700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b003d9-375f-4a40-a48c-79bd851e7472 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.508242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a961a-29ec-4ecc-a2ba-121c2ffdfc53 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.541973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.561106 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.578634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade2ba2-9985-4615-a1f0-a6ef56171535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.579450 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade2ba2-9985-4615-a1f0-a6ef56171535 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.579845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade2ba2-9985-4615-a1f0-a6ef56171535 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.580214 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade2ba2-9985-4615-a1f0-a6ef56171535 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.582403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade2ba2-9985-4615-a1f0-a6ef56171535 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:19.582403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade2ba2-9985-4615-a1f0-a6ef56171535 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.583764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade2ba2-9985-4615-a1f0-a6ef56171535 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.610472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 348/681] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:19.612208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc255efb-7dea-4406-b2fb-a7b3259a8ce2 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.612208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc255efb-7dea-4406-b2fb-a7b3259a8ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.613634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc255efb-7dea-4406-b2fb-a7b3259a8ce2 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.617354 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b003d9-375f-4a40-a48c-79bd851e7472 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.620387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 349/682] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:19.620794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc255efb-7dea-4406-b2fb-a7b3259a8ce2 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.660531 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a03ae9-0bed-473f-b2f3-29fd1543ad9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.661122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a03ae9-0bed-473f-b2f3-29fd1543ad9c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.661432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a03ae9-0bed-473f-b2f3-29fd1543ad9c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.661678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a03ae9-0bed-473f-b2f3-29fd1543ad9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.670193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a03ae9-0bed-473f-b2f3-29fd1543ad9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.672541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a03ae9-0bed-473f-b2f3-29fd1543ad9c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-54a03ae9-0bed-473f-b2f3-29fd1543ad9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:19.686664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:19.687683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:19.688565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:19.688565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:19.688565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:19.688565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:19.688565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:19.688565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:19.688565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:19.688565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:19.688565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 334/683] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:19.689791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f14ce-ea8d-4dc3-b9aa-72bba0d9fb68 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.690076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f14ce-ea8d-4dc3-b9aa-72bba0d9fb68 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.690343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f14ce-ea8d-4dc3-b9aa-72bba0d9fb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.716063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1f0f14ce-ea8d-4dc3-b9aa-72bba0d9fb68 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:19.717327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1f0f14ce-ea8d-4dc3-b9aa-72bba0d9fb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:19.766083 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc255efb-7dea-4406-b2fb-a7b3259a8ce2 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.803735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34976f6a-c72e-4eeb-8cf6-ddd142c57e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.804400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34976f6a-c72e-4eeb-8cf6-ddd142c57e80 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.804696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34976f6a-c72e-4eeb-8cf6-ddd142c57e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.804971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34976f6a-c72e-4eeb-8cf6-ddd142c57e80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.806446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 335/684] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:19.807804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f14ce-ea8d-4dc3-b9aa-72bba0d9fb68 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.857965 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.869079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.876894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34976f6a-c72e-4eeb-8cf6-ddd142c57e80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.878503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34976f6a-c72e-4eeb-8cf6-ddd142c57e80 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.984731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 350/685] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:19.993737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34976f6a-c72e-4eeb-8cf6-ddd142c57e80 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.998081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f14ce-ea8d-4dc3-b9aa-72bba0d9fb68 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.036814 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6ee99-75b2-44c5-beeb-863c4ec2b5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.037420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6ee99-75b2-44c5-beeb-863c4ec2b5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.037695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6ee99-75b2-44c5-beeb-863c4ec2b5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.037961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6ee99-75b2-44c5-beeb-863c4ec2b5a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.038658 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6ee99-75b2-44c5-beeb-863c4ec2b5a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:20.038954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6ee99-75b2-44c5-beeb-863c4ec2b5a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.040415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6ee99-75b2-44c5-beeb-863c4ec2b5a0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.063335 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 336/686] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:20.070013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6ee99-75b2-44c5-beeb-863c4ec2b5a0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.158761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.179371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34976f6a-c72e-4eeb-8cf6-ddd142c57e80 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.210324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.215545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6ee99-75b2-44c5-beeb-863c4ec2b5a0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.227767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf25ad-138c-41ff-9b0d-b8a084ced125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.228409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf25ad-138c-41ff-9b0d-b8a084ced125 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.228724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf25ad-138c-41ff-9b0d-b8a084ced125 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.229004 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf25ad-138c-41ff-9b0d-b8a084ced125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.241533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf25ad-138c-41ff-9b0d-b8a084ced125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.244171 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf25ad-138c-41ff-9b0d-b8a084ced125 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.258756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a55f3-82cc-4bfe-b58a-af0a8ddf4fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.259498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a55f3-82cc-4bfe-b58a-af0a8ddf4fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.259871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a55f3-82cc-4bfe-b58a-af0a8ddf4fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.260241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a55f3-82cc-4bfe-b58a-af0a8ddf4fc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.261128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a55f3-82cc-4bfe-b58a-af0a8ddf4fc4 None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=5e18461c8cc4497eae7a83d9ad1448f6) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:20.261608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a55f3-82cc-4bfe-b58a-af0a8ddf4fc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.263262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a55f3-82cc-4bfe-b58a-af0a8ddf4fc4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-c6bf25ad-138c-41ff-9b0d-b8a084ced125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:20.264619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:20.265735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:20.266807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:20.266807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:20.266807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:20.271904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 337/688] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/role_assignments?user.id=5e18461c8cc4497eae7a83d9ad1448f6&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:20.272093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 351/688] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:20.273403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a55f3-82cc-4bfe-b58a-af0a8ddf4fc4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.274859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf25ad-138c-41ff-9b0d-b8a084ced125 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.388399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a55f3-82cc-4bfe-b58a-af0a8ddf4fc4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.400863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf25ad-138c-41ff-9b0d-b8a084ced125 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.427254 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506642d-5327-4b66-9017-8c281e3ee448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.428150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506642d-5327-4b66-9017-8c281e3ee448 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.428563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506642d-5327-4b66-9017-8c281e3ee448 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.429013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506642d-5327-4b66-9017-8c281e3ee448 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.438654 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef58385-9939-41f6-9813-a4d43df1c973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.439437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef58385-9939-41f6-9813-a4d43df1c973 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.439814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef58385-9939-41f6-9813-a4d43df1c973 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.440191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef58385-9939-41f6-9813-a4d43df1c973 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.440982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef58385-9939-41f6-9813-a4d43df1c973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:20.441377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef58385-9939-41f6-9813-a4d43df1c973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.442058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506642d-5327-4b66-9017-8c281e3ee448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:20.442988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef58385-9939-41f6-9813-a4d43df1c973 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-4506642d-5327-4b66-9017-8c281e3ee448 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:20.447665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:20.448779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:20.449631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:20.449631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:20.449631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:20.449631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:20.449631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:20.449631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:20.449631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:20.449631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:20.449631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:20.449631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:20.449631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:20.449631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:20.452199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 338/689] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:20.452586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.465807 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506642d-5327-4b66-9017-8c281e3ee448 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.468306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 352/690] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:20.471441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef58385-9939-41f6-9813-a4d43df1c973 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.516685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.582477 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef58385-9939-41f6-9813-a4d43df1c973 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.624192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86547bd-5ea2-4979-aa55-3a8eb05d3244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.624731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86547bd-5ea2-4979-aa55-3a8eb05d3244 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.625020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86547bd-5ea2-4979-aa55-3a8eb05d3244 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.625287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86547bd-5ea2-4979-aa55-3a8eb05d3244 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.643581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506642d-5327-4b66-9017-8c281e3ee448 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.645764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86547bd-5ea2-4979-aa55-3a8eb05d3244 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.647251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86547bd-5ea2-4979-aa55-3a8eb05d3244 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-d86547bd-5ea2-4979-aa55-3a8eb05d3244 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:20.672583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:20.673552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:20.674423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:20.674423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:20.674423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:20.674423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:20.674423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:20.674423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:20.674423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:20.674423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:20.679392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 353/691] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:20.684070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86547bd-5ea2-4979-aa55-3a8eb05d3244 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.694463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c31aa-47f4-47a3-a888-9a6eee077c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.695390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c31aa-47f4-47a3-a888-9a6eee077c79 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.695859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c31aa-47f4-47a3-a888-9a6eee077c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.696312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c31aa-47f4-47a3-a888-9a6eee077c79 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.697244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c31aa-47f4-47a3-a888-9a6eee077c79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:20.697779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c31aa-47f4-47a3-a888-9a6eee077c79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.699470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c31aa-47f4-47a3-a888-9a6eee077c79 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.709201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 339/692] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:20.716727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c31aa-47f4-47a3-a888-9a6eee077c79 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.754123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.802703 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86547bd-5ea2-4979-aa55-3a8eb05d3244 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.828245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.846819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50030278-9c9a-4907-bb65-af180f24287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.847561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50030278-9c9a-4907-bb65-af180f24287b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.847931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50030278-9c9a-4907-bb65-af180f24287b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.848291 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50030278-9c9a-4907-bb65-af180f24287b None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.848720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c31aa-47f4-47a3-a888-9a6eee077c79 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.849321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50030278-9c9a-4907-bb65-af180f24287b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:20.849715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50030278-9c9a-4907-bb65-af180f24287b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.851455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50030278-9c9a-4907-bb65-af180f24287b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.872494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 354/693] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:20.874262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50030278-9c9a-4907-bb65-af180f24287b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.894006 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bee5af-bcbc-473c-98d4-b0bfacd15fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.894691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bee5af-bcbc-473c-98d4-b0bfacd15fee None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.894969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bee5af-bcbc-473c-98d4-b0bfacd15fee None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.895294 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bee5af-bcbc-473c-98d4-b0bfacd15fee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.908924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bee5af-bcbc-473c-98d4-b0bfacd15fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.910481 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bee5af-bcbc-473c-98d4-b0bfacd15fee None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-89bee5af-bcbc-473c-98d4-b0bfacd15fee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:20.927186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:20.928188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:20.929551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:20.929551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:20.929551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:20.929551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 340/694] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:20.934581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bee5af-bcbc-473c-98d4-b0bfacd15fee None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:21.001407 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50030278-9c9a-4907-bb65-af180f24287b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:21.030145 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:21.067614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bee5af-bcbc-473c-98d4-b0bfacd15fee None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:21.076824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd51bd-f1a8-4188-b94b-880dd11f142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:21.077505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd51bd-f1a8-4188-b94b-880dd11f142c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.077797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd51bd-f1a8-4188-b94b-880dd11f142c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.078059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd51bd-f1a8-4188-b94b-880dd11f142c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.079025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd51bd-f1a8-4188-b94b-880dd11f142c None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=bfec040b09c9415b9e89cd28c67aa5df) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:21.079134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd51bd-f1a8-4188-b94b-880dd11f142c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:21.080763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd51bd-f1a8-4188-b94b-880dd11f142c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:21.098935 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 355/695] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/role_assignments?user.id=bfec040b09c9415b9e89cd28c67aa5df&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:21.114021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c1326-ed8c-408b-9deb-efb7fa4add7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:21.115036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c1326-ed8c-408b-9deb-efb7fa4add7a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.115529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c1326-ed8c-408b-9deb-efb7fa4add7a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.116025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c1326-ed8c-408b-9deb-efb7fa4add7a None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.116931 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c1326-ed8c-408b-9deb-efb7fa4add7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:21.117463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c1326-ed8c-408b-9deb-efb7fa4add7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:21.119311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c1326-ed8c-408b-9deb-efb7fa4add7a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:21.137325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:21.144690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 341/696] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:21.151470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd51bd-f1a8-4188-b94b-880dd11f142c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:21.294353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd51bd-f1a8-4188-b94b-880dd11f142c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:21.329521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c200d3-1f94-4a08-85ee-d775ecd1b3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:21.330156 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c200d3-1f94-4a08-85ee-d775ecd1b3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.330430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c200d3-1f94-4a08-85ee-d775ecd1b3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.330701 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c200d3-1f94-4a08-85ee-d775ecd1b3d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.339335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c200d3-1f94-4a08-85ee-d775ecd1b3d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:21.341029 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c200d3-1f94-4a08-85ee-d775ecd1b3d4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-b5c200d3-1f94-4a08-85ee-d775ecd1b3d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:21.352805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:21.353797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:21.355594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:21.355594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:21.355594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:21.355594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:21.355594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:21.355594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:21.355594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:21.355594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:21.355594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:21.355594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:21.355594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 356/697] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 12:20:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:21.360299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c1326-ed8c-408b-9deb-efb7fa4add7a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:21.468301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c1326-ed8c-408b-9deb-efb7fa4add7a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:21.505059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98862c6c-702f-4bd5-94ce-4051e85ab4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:21.506038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98862c6c-702f-4bd5-94ce-4051e85ab4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.506559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98862c6c-702f-4bd5-94ce-4051e85ab4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.507065 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98862c6c-702f-4bd5-94ce-4051e85ab4b0 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.507972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98862c6c-702f-4bd5-94ce-4051e85ab4b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:21.508462 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98862c6c-702f-4bd5-94ce-4051e85ab4b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:21.510163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98862c6c-702f-4bd5-94ce-4051e85ab4b0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:21.530243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 342/698] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 12:20:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:21.536166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c200d3-1f94-4a08-85ee-d775ecd1b3d4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:21.581970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:21.642526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c200d3-1f94-4a08-85ee-d775ecd1b3d4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:21.679854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfdf48-bc42-4357-a007-4f270e24c588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:21.680471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfdf48-bc42-4357-a007-4f270e24c588 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.680778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfdf48-bc42-4357-a007-4f270e24c588 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.681049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfdf48-bc42-4357-a007-4f270e24c588 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.697866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52959b53-51e0-491e-b79a-0ecb18fe7fe0 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.698303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52959b53-51e0-491e-b79a-0ecb18fe7fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.698652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52959b53-51e0-491e-b79a-0ecb18fe7fe0 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.700674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 343/699] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:21.706281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76cde-25e5-4d6e-816f-ac8c3ecef97c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.706562 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76cde-25e5-4d6e-816f-ac8c3ecef97c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.706832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76cde-25e5-4d6e-816f-ac8c3ecef97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.729166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-fff76cde-25e5-4d6e-816f-ac8c3ecef97c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:21.730406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fff76cde-25e5-4d6e-816f-ac8c3ecef97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:21.765975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfdf48-bc42-4357-a007-4f270e24c588 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:21.767501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfdf48-bc42-4357-a007-4f270e24c588 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:21.812289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:21.815966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:21.853068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 344/700] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:20:21.880433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 357/701] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:21.885755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff76cde-25e5-4d6e-816f-ac8c3ecef97c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:21.894565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc7fba4-ab28-4d7f-9211-0102021c8e7d None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.894838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc7fba4-ab28-4d7f-9211-0102021c8e7d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.895110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc7fba4-ab28-4d7f-9211-0102021c8e7d None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.896552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 358/702] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:21.903100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc3963-e498-4f76-b89b-71096afda061 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.903383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc3963-e498-4f76-b89b-71096afda061 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.903637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc3963-e498-4f76-b89b-71096afda061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.924332 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5edc3963-e498-4f76-b89b-71096afda061 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:21.925484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5edc3963-e498-4f76-b89b-71096afda061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:22.000009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 359/703] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:20:22.064393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff76cde-25e5-4d6e-816f-ac8c3ecef97c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:22.100765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4fbdb-36b6-408b-97b6-f60d4180530e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:22.101395 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4fbdb-36b6-408b-97b6-f60d4180530e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.101674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4fbdb-36b6-408b-97b6-f60d4180530e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.101932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4fbdb-36b6-408b-97b6-f60d4180530e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.113568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4fbdb-36b6-408b-97b6-f60d4180530e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:22.114966 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4fbdb-36b6-408b-97b6-f60d4180530e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b4b4fbdb-36b6-408b-97b6-f60d4180530e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:22.130115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:22.131097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:22.132098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:22.132098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:22.132098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:22.132098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 345/704] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:22.137528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc3963-e498-4f76-b89b-71096afda061 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:22.192041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4fbdb-36b6-408b-97b6-f60d4180530e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:22.219304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:22.248126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc3963-e498-4f76-b89b-71096afda061 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:22.284357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b9a50-6d56-4317-9cad-386801202bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:22.285258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b9a50-6d56-4317-9cad-386801202bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.285539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b9a50-6d56-4317-9cad-386801202bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.285806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b9a50-6d56-4317-9cad-386801202bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.286494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b9a50-6d56-4317-9cad-386801202bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:22.286839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b9a50-6d56-4317-9cad-386801202bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:22.288231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b9a50-6d56-4317-9cad-386801202bf3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:22.298267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:22.304924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 360/705] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:22.312387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b9a50-6d56-4317-9cad-386801202bf3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:22.421605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b9a50-6d56-4317-9cad-386801202bf3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:22.427192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4fbdb-36b6-408b-97b6-f60d4180530e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:22.459742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:22.467385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb295716-d1e2-4962-bf58-6a3ed54584da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:22.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb295716-d1e2-4962-bf58-6a3ed54584da None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.468835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb295716-d1e2-4962-bf58-6a3ed54584da None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.469252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb295716-d1e2-4962-bf58-6a3ed54584da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.474548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b28383-f5fb-420e-9019-4c4ee3d67c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:22.475293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b28383-f5fb-420e-9019-4c4ee3d67c73 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.475659 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b28383-f5fb-420e-9019-4c4ee3d67c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.476020 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b28383-f5fb-420e-9019-4c4ee3d67c73 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.476721 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b28383-f5fb-420e-9019-4c4ee3d67c73 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:22.477824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb295716-d1e2-4962-bf58-6a3ed54584da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:22.478284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b28383-f5fb-420e-9019-4c4ee3d67c73 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:22.479587 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb295716-d1e2-4962-bf58-6a3ed54584da None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-53b28383-f5fb-420e-9019-4c4ee3d67c73 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:22.491307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:22.492432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:22.493311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 346/706] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Aug 24 12:20:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:22.494088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b28383-f5fb-420e-9019-4c4ee3d67c73 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-fb295716-d1e2-4962-bf58-6a3ed54584da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:22.499128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:22.500027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:22.501500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:22.501500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:22.501500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:22.501500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:22.501500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:22.501500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:22.501500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:22.501500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:22.501500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 361/707] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:22.501500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb295716-d1e2-4962-bf58-6a3ed54584da None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:22.613131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b28383-f5fb-420e-9019-4c4ee3d67c73 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:22.614669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:22.620244 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb295716-d1e2-4962-bf58-6a3ed54584da None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:22.653678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bc63d-451e-4c18-b525-ab4242068a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:22.654713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bc63d-451e-4c18-b525-ab4242068a15 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.655167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bc63d-451e-4c18-b525-ab4242068a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.655579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bc63d-451e-4c18-b525-ab4242068a15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.658164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab526ad-152b-4667-bc67-49c9844a262a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:22.658868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab526ad-152b-4667-bc67-49c9844a262a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.659431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab526ad-152b-4667-bc67-49c9844a262a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.659681 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab526ad-152b-4667-bc67-49c9844a262a None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.660401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab526ad-152b-4667-bc67-49c9844a262a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:22.660829 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab526ad-152b-4667-bc67-49c9844a262a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:22.662286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab526ad-152b-4667-bc67-49c9844a262a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:22.667900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 362/708] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 24 12:20:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:22.669461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab526ad-152b-4667-bc67-49c9844a262a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:22.671940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bc63d-451e-4c18-b525-ab4242068a15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:22.673424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bc63d-451e-4c18-b525-ab4242068a15 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b89bc63d-451e-4c18-b525-ab4242068a15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:22.689361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:22.690338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:22.691232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:22.691232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:22.691232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:22.695289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 347/709] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:22.695901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bc63d-451e-4c18-b525-ab4242068a15 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:22.747364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:22.806549 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab526ad-152b-4667-bc67-49c9844a262a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:22.808247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bc63d-451e-4c18-b525-ab4242068a15 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:22.853307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360fb70-8bc9-4b28-94cc-0329eaa37a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:22.853937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360fb70-8bc9-4b28-94cc-0329eaa37a26 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.854278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360fb70-8bc9-4b28-94cc-0329eaa37a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.854474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360fb70-8bc9-4b28-94cc-0329eaa37a26 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.855184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6360fb70-8bc9-4b28-94cc-0329eaa37a26 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:22.855889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6360fb70-8bc9-4b28-94cc-0329eaa37a26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:22.857060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6360fb70-8bc9-4b28-94cc-0329eaa37a26 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:22.863934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb971e0d-db78-4c55-9323-f776077f24b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:22.864239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 348/710] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 24 12:20:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:22.865087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb971e0d-db78-4c55-9323-f776077f24b4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.865210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360fb70-8bc9-4b28-94cc-0329eaa37a26 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:22.865678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb971e0d-db78-4c55-9323-f776077f24b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.866147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb971e0d-db78-4c55-9323-f776077f24b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.867088 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb971e0d-db78-4c55-9323-f776077f24b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:22.867601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb971e0d-db78-4c55-9323-f776077f24b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:22.869326 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb971e0d-db78-4c55-9323-f776077f24b4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:22.890508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 363/711] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:22.892582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:22.896755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb971e0d-db78-4c55-9323-f776077f24b4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:23.025622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360fb70-8bc9-4b28-94cc-0329eaa37a26 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:23.033969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb971e0d-db78-4c55-9323-f776077f24b4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:23.052149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:23.073463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e291328-9814-436e-a860-4ad43bf6b877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:23.074793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e291328-9814-436e-a860-4ad43bf6b877 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:23.075070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e291328-9814-436e-a860-4ad43bf6b877 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:23.075340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e291328-9814-436e-a860-4ad43bf6b877 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.076001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e291328-9814-436e-a860-4ad43bf6b877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:23.079158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e291328-9814-436e-a860-4ad43bf6b877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:23.079158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e291328-9814-436e-a860-4ad43bf6b877 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:23.079423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e0884-24bc-415e-8c7b-5e58c19a0a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:23.079423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e0884-24bc-415e-8c7b-5e58c19a0a64 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:23.079423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e0884-24bc-415e-8c7b-5e58c19a0a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:23.079423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e0884-24bc-415e-8c7b-5e58c19a0a64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.079792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e0884-24bc-415e-8c7b-5e58c19a0a64 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=e65026e01df242e7be11ed24214a4efa) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:23.080078 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e0884-24bc-415e-8c7b-5e58c19a0a64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:23.081556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e0884-24bc-415e-8c7b-5e58c19a0a64 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:23.087632 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 364/712] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:22 2022] GET /identity/v3/role_assignments?user.id=e65026e01df242e7be11ed24214a4efa&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:23.089309 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef08a23-bf9b-4bce-9bed-92877e554f1d None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:23.089840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef08a23-bf9b-4bce-9bed-92877e554f1d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:23.090595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef08a23-bf9b-4bce-9bed-92877e554f1d None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.092126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 365/713] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:23.096799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 349/714] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:23.102316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068557fb-8e83-409a-8c47-40eb71aeda60 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:23.103624 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e291328-9814-436e-a860-4ad43bf6b877 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:23.105158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068557fb-8e83-409a-8c47-40eb71aeda60 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:23.107321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068557fb-8e83-409a-8c47-40eb71aeda60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.142338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-068557fb-8e83-409a-8c47-40eb71aeda60 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:23.148255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-068557fb-8e83-409a-8c47-40eb71aeda60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:23.255476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:23.283954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e291328-9814-436e-a860-4ad43bf6b877 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:23.291150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 366/715] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:23.325925 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cbe8e-84cf-4fde-a1b3-5560f318c9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:23.326879 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cbe8e-84cf-4fde-a1b3-5560f318c9a6 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:23.327352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cbe8e-84cf-4fde-a1b3-5560f318c9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:23.327838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cbe8e-84cf-4fde-a1b3-5560f318c9a6 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.328730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8cbe8e-84cf-4fde-a1b3-5560f318c9a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:23.330369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8cbe8e-84cf-4fde-a1b3-5560f318c9a6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:23.359717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 350/716] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:20:23 2022] POST /identity/v3/users => generated 254 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:23.653186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068557fb-8e83-409a-8c47-40eb71aeda60 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:23.758041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068557fb-8e83-409a-8c47-40eb71aeda60 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:23.769867 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:23.798633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb263d-3b69-4fc8-a164-a66395fca430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:23.799285 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb263d-3b69-4fc8-a164-a66395fca430 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:23.799561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb263d-3b69-4fc8-a164-a66395fca430 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:23.799830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb263d-3b69-4fc8-a164-a66395fca430 None admin] PATH_INFO: `/v3/services/network` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.800417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb263d-3b69-4fc8-a164-a66395fca430 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:23.801852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb263d-3b69-4fc8-a164-a66395fca430 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-98eb263d-3b69-4fc8-a164-a66395fca430 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:23.819892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:23.821015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:23.822461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 367/717] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:23 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:23.827891 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cbe8e-84cf-4fde-a1b3-5560f318c9a6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:23.976123 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cbe8e-84cf-4fde-a1b3-5560f318c9a6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:24.035363 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b3bb4-22ea-4d7c-b129-233827677381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:24.036387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b3bb4-22ea-4d7c-b129-233827677381 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.036910 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b3bb4-22ea-4d7c-b129-233827677381 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.037389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b3bb4-22ea-4d7c-b129-233827677381 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.038730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b3bb4-22ea-4d7c-b129-233827677381 None admin] RBAC: Adding query filter params (name=network) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:24.039406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b3bb4-22ea-4d7c-b129-233827677381 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:24.041741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b3bb4-22ea-4d7c-b129-233827677381 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:24.041859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b760908-2ac6-413c-b4a7-251aae09d8d4 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.042431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b760908-2ac6-413c-b4a7-251aae09d8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.042869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b760908-2ac6-413c-b4a7-251aae09d8d4 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.043809 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:24.045897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 368/718] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:24.051225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 351/719] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:24.054171 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aed79-f920-46a5-a27d-341b08e7af38 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.054657 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aed79-f920-46a5-a27d-341b08e7af38 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.055236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8aed79-f920-46a5-a27d-341b08e7af38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.058936 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b3bb4-22ea-4d7c-b129-233827677381 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:24.123938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6a8aed79-f920-46a5-a27d-341b08e7af38 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:24.125802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6a8aed79-f920-46a5-a27d-341b08e7af38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:24.237958 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b3bb4-22ea-4d7c-b129-233827677381 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:24.257378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 369/720] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:20:24.297156 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed81931-9477-4ac9-99cb-f9f60606558f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:24.298520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed81931-9477-4ac9-99cb-f9f60606558f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.299238 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed81931-9477-4ac9-99cb-f9f60606558f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.299934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed81931-9477-4ac9-99cb-f9f60606558f None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.301524 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed81931-9477-4ac9-99cb-f9f60606558f None admin] RBAC: Adding query filter params (type=network) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:24.302306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed81931-9477-4ac9-99cb-f9f60606558f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:24.305132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed81931-9477-4ac9-99cb-f9f60606558f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:24.315754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 352/721] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:24.699151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aed79-f920-46a5-a27d-341b08e7af38 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:24.881279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8aed79-f920-46a5-a27d-341b08e7af38 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:24.889695 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:24.929372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbec55b-0253-49d2-97eb-b960bac864e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:24.930022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbec55b-0253-49d2-97eb-b960bac864e1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.930298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbec55b-0253-49d2-97eb-b960bac864e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.930567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbec55b-0253-49d2-97eb-b960bac864e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.942459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbec55b-0253-49d2-97eb-b960bac864e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-7cbec55b-0253-49d2-97eb-b960bac864e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:24.947256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:24.948182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:24.949827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 370/722] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:24.959275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed81931-9477-4ac9-99cb-f9f60606558f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.074100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:25.127490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed81931-9477-4ac9-99cb-f9f60606558f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.174263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e2bd9-1a6b-493e-b2ed-8dc0c0f3645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.174880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e2bd9-1a6b-493e-b2ed-8dc0c0f3645b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.175161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e2bd9-1a6b-493e-b2ed-8dc0c0f3645b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.175433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e2bd9-1a6b-493e-b2ed-8dc0c0f3645b None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.176109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e2bd9-1a6b-493e-b2ed-8dc0c0f3645b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:25.176411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e2bd9-1a6b-493e-b2ed-8dc0c0f3645b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.177865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e2bd9-1a6b-493e-b2ed-8dc0c0f3645b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.192676 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 353/723] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:25.198265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbec55b-0253-49d2-97eb-b960bac864e1 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.354652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75a829-2d36-4175-8206-a8fbb0fcebcf None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.355034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75a829-2d36-4175-8206-a8fbb0fcebcf None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.355460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75a829-2d36-4175-8206-a8fbb0fcebcf None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.357688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 354/724] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:25.366043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1241c2b-8334-4648-8535-4796b1973c84 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.366448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1241c2b-8334-4648-8535-4796b1973c84 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.366824 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1241c2b-8334-4648-8535-4796b1973c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.367561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbec55b-0253-49d2-97eb-b960bac864e1 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.397896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f1241c2b-8334-4648-8535-4796b1973c84 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:25.399258 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f1241c2b-8334-4648-8535-4796b1973c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:25.412016 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf8ffa-6b14-43cf-9ea2-51d9ca3508bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.412897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf8ffa-6b14-43cf-9ea2-51d9ca3508bb None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.413323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf8ffa-6b14-43cf-9ea2-51d9ca3508bb None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.413763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf8ffa-6b14-43cf-9ea2-51d9ca3508bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.431544 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf8ffa-6b14-43cf-9ea2-51d9ca3508bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.433851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf8ffa-6b14-43cf-9ea2-51d9ca3508bb None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-bdbf8ffa-6b14-43cf-9ea2-51d9ca3508bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:25.459942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:25.461206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:25.462144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:25.462144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:25.462144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:25.462144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 371/725] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:25.468338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf8ffa-6b14-43cf-9ea2-51d9ca3508bb None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.509555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:25.511180 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 355/726] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:25.661422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a86e00-2876-4b8c-8b92-800316386819 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.661619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a86e00-2876-4b8c-8b92-800316386819 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.661870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a86e00-2876-4b8c-8b92-800316386819 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.663466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 356/727] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:25.669154 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46046d-0d9b-422e-92a1-530393377c2d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.669444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46046d-0d9b-422e-92a1-530393377c2d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.669695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46046d-0d9b-422e-92a1-530393377c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.716895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-fd46046d-0d9b-422e-92a1-530393377c2d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:25.718134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fd46046d-0d9b-422e-92a1-530393377c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:25.744738 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf8ffa-6b14-43cf-9ea2-51d9ca3508bb None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.817984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6eaea-d279-4293-bb09-df0bc1c17c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.818603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6eaea-d279-4293-bb09-df0bc1c17c90 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.818873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6eaea-d279-4293-bb09-df0bc1c17c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.819143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6eaea-d279-4293-bb09-df0bc1c17c90 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.819801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6eaea-d279-4293-bb09-df0bc1c17c90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:25.820087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6eaea-d279-4293-bb09-df0bc1c17c90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.821499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6eaea-d279-4293-bb09-df0bc1c17c90 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.826764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:25.845258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 372/728] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:25.850391 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6eaea-d279-4293-bb09-df0bc1c17c90 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.859249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 357/729] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:20:25.861083 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46046d-0d9b-422e-92a1-530393377c2d None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.983786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46046d-0d9b-422e-92a1-530393377c2d None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.996715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.028169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3bf3a-0fe5-4aa3-91f0-d6f16b1667b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.029210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3bf3a-0fe5-4aa3-91f0-d6f16b1667b0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.029758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3bf3a-0fe5-4aa3-91f0-d6f16b1667b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.030228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3bf3a-0fe5-4aa3-91f0-d6f16b1667b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.038466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3bf3a-0fe5-4aa3-91f0-d6f16b1667b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.040174 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3bf3a-0fe5-4aa3-91f0-d6f16b1667b0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.043116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6eaea-d279-4293-bb09-df0bc1c17c90 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3cb3bf3a-0fe5-4aa3-91f0-d6f16b1667b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:26.053828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:26.054792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:26.055735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:26.055735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:26.055735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:26.055735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:26.055735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:26.055735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:26.055735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:26.055735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:26.055735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:26.055735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:26.058382 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 358/730] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:26.071789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3bf3a-0fe5-4aa3-91f0-d6f16b1667b0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.089457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41567f-f19f-4ad1-9439-96d112578b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.090185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41567f-f19f-4ad1-9439-96d112578b13 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.090437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41567f-f19f-4ad1-9439-96d112578b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.090700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41567f-f19f-4ad1-9439-96d112578b13 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.091341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41567f-f19f-4ad1-9439-96d112578b13 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.092735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41567f-f19f-4ad1-9439-96d112578b13 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.119700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 373/731] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:25 2022] POST /identity/v3/services => generated 251 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:26.190201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.264595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3bf3a-0fe5-4aa3-91f0-d6f16b1667b0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.268270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0e8b2-0574-4708-a091-d170c5843153 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.268547 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0e8b2-0574-4708-a091-d170c5843153 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.268843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0e8b2-0574-4708-a091-d170c5843153 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.270313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 374/732] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:26.270851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0e8b2-0574-4708-a091-d170c5843153 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.311902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b13cdd-7065-4973-9440-304bd146eb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.312530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b13cdd-7065-4973-9440-304bd146eb93 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.312837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b13cdd-7065-4973-9440-304bd146eb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.313109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b13cdd-7065-4973-9440-304bd146eb93 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.313856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b13cdd-7065-4973-9440-304bd146eb93 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:26.314162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b13cdd-7065-4973-9440-304bd146eb93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.315626 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b13cdd-7065-4973-9440-304bd146eb93 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.338312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 359/733] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:26.340777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d6722-1482-43b4-b221-6cbb6f9da488 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.341141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d6722-1482-43b4-b221-6cbb6f9da488 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.341415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d6722-1482-43b4-b221-6cbb6f9da488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.361621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-281d6722-1482-43b4-b221-6cbb6f9da488 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:26.362853 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-281d6722-1482-43b4-b221-6cbb6f9da488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:26.420167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0e8b2-0574-4708-a091-d170c5843153 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.444225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 360/734] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:26.445133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d6722-1482-43b4-b221-6cbb6f9da488 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.464339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0f1c1-4fb7-4cfa-adaa-abd87d70ddcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.465645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0f1c1-4fb7-4cfa-adaa-abd87d70ddcb None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.466191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0f1c1-4fb7-4cfa-adaa-abd87d70ddcb None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.466705 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0f1c1-4fb7-4cfa-adaa-abd87d70ddcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.470961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.481744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.481923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b0f1c1-4fb7-4cfa-adaa-abd87d70ddcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-e1b0f1c1-4fb7-4cfa-adaa-abd87d70ddcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:26.489093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:26.490064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:26.490891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 375/735] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:26.496950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0f1c1-4fb7-4cfa-adaa-abd87d70ddcb None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.564553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d6722-1482-43b4-b221-6cbb6f9da488 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.621475 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0f1c1-4fb7-4cfa-adaa-abd87d70ddcb None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.653480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf209902-7831-47b4-aaa1-19aabad87ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.654114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf209902-7831-47b4-aaa1-19aabad87ecc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.654393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf209902-7831-47b4-aaa1-19aabad87ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.654667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf209902-7831-47b4-aaa1-19aabad87ecc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.659337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aa57d-ddb0-43be-bcee-ea71aaecf3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.659944 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aa57d-ddb0-43be-bcee-ea71aaecf3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.660218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aa57d-ddb0-43be-bcee-ea71aaecf3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.660485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49aa57d-ddb0-43be-bcee-ea71aaecf3e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.661173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49aa57d-ddb0-43be-bcee-ea71aaecf3e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:26.661470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49aa57d-ddb0-43be-bcee-ea71aaecf3e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.663143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49aa57d-ddb0-43be-bcee-ea71aaecf3e3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.674366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 376/736] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:26.680152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aa57d-ddb0-43be-bcee-ea71aaecf3e3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.748693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.787663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf209902-7831-47b4-aaa1-19aabad87ecc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.789935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf209902-7831-47b4-aaa1-19aabad87ecc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.797883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49aa57d-ddb0-43be-bcee-ea71aaecf3e3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.821748 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.835338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a5407-7697-4c3c-8171-de40d07dea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.835969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a5407-7697-4c3c-8171-de40d07dea60 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.836239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a5407-7697-4c3c-8171-de40d07dea60 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.836500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a5407-7697-4c3c-8171-de40d07dea60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.847702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a5407-7697-4c3c-8171-de40d07dea60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.849297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a5407-7697-4c3c-8171-de40d07dea60 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-905a5407-7697-4c3c-8171-de40d07dea60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:26.863331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:26.864390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:26.865336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:26.865336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:26.865336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:26.865336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 377/737] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:26.865336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a5407-7697-4c3c-8171-de40d07dea60 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.915102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 361/738] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:26.916491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bacbad-68a5-4a7c-9e6b-3007e6d4e05a None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.916881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bacbad-68a5-4a7c-9e6b-3007e6d4e05a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.917348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bacbad-68a5-4a7c-9e6b-3007e6d4e05a None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.923521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 362/739] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:26.923521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bacbad-68a5-4a7c-9e6b-3007e6d4e05a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.992706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a5407-7697-4c3c-8171-de40d07dea60 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.008042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:27.032559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bacbad-68a5-4a7c-9e6b-3007e6d4e05a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.038584 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12064b-4ba1-45db-992e-5cb9ac8ec35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.039449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12064b-4ba1-45db-992e-5cb9ac8ec35e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.040337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12064b-4ba1-45db-992e-5cb9ac8ec35e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.040788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12064b-4ba1-45db-992e-5cb9ac8ec35e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.054634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c12064b-4ba1-45db-992e-5cb9ac8ec35e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:27.059622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-2c12064b-4ba1-45db-992e-5cb9ac8ec35e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:27.061288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:27.062504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:27.063472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:27.063472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:27.063472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:27.063472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:27.063472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:27.063472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:27.063472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:27.063472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:27.063472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:27.063472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:27.063472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:27.063472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:27.065755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 378/740] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:27.066607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12064b-4ba1-45db-992e-5cb9ac8ec35e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.071926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8f79c-7233-4578-ad10-10b0c4ad37d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.072470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8f79c-7233-4578-ad10-10b0c4ad37d6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.072766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8f79c-7233-4578-ad10-10b0c4ad37d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.073031 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8f79c-7233-4578-ad10-10b0c4ad37d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.073784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8f79c-7233-4578-ad10-10b0c4ad37d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:27.074352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8f79c-7233-4578-ad10-10b0c4ad37d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.075475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8f79c-7233-4578-ad10-10b0c4ad37d6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.087538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 363/741] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:27.089178 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5711c-3528-4c65-8a09-760b00ee0b7b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.089561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5711c-3528-4c65-8a09-760b00ee0b7b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.090021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5711c-3528-4c65-8a09-760b00ee0b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.110433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6db5711c-3528-4c65-8a09-760b00ee0b7b None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:27.111615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6db5711c-3528-4c65-8a09-760b00ee0b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:27.180306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 364/742] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:27.181177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5711c-3528-4c65-8a09-760b00ee0b7b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.192265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12064b-4ba1-45db-992e-5cb9ac8ec35e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.235997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c5f54-4e2b-4d6b-934b-25887fb121ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.236622 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c5f54-4e2b-4d6b-934b-25887fb121ca None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.236928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c5f54-4e2b-4d6b-934b-25887fb121ca None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.237191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c5f54-4e2b-4d6b-934b-25887fb121ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.248360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c5f54-4e2b-4d6b-934b-25887fb121ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.249886 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c5f54-4e2b-4d6b-934b-25887fb121ca None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-7a9c5f54-4e2b-4d6b-934b-25887fb121ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:27.264616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:27.265571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:27.266446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:27.266446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:27.266446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:27.270376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 379/743] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:27.271260 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c5f54-4e2b-4d6b-934b-25887fb121ca None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.304883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:27.311575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5711c-3528-4c65-8a09-760b00ee0b7b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.337181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:27.353209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a698b-01ed-41f8-80fc-1eda3b2da88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.353907 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a698b-01ed-41f8-80fc-1eda3b2da88c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.354404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a698b-01ed-41f8-80fc-1eda3b2da88c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.354789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a698b-01ed-41f8-80fc-1eda3b2da88c None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.356062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a698b-01ed-41f8-80fc-1eda3b2da88c None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:27.356425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a698b-01ed-41f8-80fc-1eda3b2da88c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.358173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a698b-01ed-41f8-80fc-1eda3b2da88c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.368174 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 365/744] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:27.369137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a698b-01ed-41f8-80fc-1eda3b2da88c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.401984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c5f54-4e2b-4d6b-934b-25887fb121ca None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.439967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d392315-55bf-42bd-9ec1-96bcf787d08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.440624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d392315-55bf-42bd-9ec1-96bcf787d08a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.441028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d392315-55bf-42bd-9ec1-96bcf787d08a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.441337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d392315-55bf-42bd-9ec1-96bcf787d08a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.450641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d392315-55bf-42bd-9ec1-96bcf787d08a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.453183 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d392315-55bf-42bd-9ec1-96bcf787d08a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-9d392315-55bf-42bd-9ec1-96bcf787d08a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:27.467279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:27.468401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:27.469488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:27.469488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:27.469488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:27.469488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:27.469488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:27.469488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:27.469488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:27.469488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:27.469488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:27.469488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:27.470076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 380/745] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:27.471033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d392315-55bf-42bd-9ec1-96bcf787d08a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.509401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a698b-01ed-41f8-80fc-1eda3b2da88c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.547061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674c54-8852-406f-89a5-78d8a45fdb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.547665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674c54-8852-406f-89a5-78d8a45fdb99 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.547957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674c54-8852-406f-89a5-78d8a45fdb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.548222 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674c54-8852-406f-89a5-78d8a45fdb99 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.548903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4674c54-8852-406f-89a5-78d8a45fdb99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:27.549207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4674c54-8852-406f-89a5-78d8a45fdb99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.550597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4674c54-8852-406f-89a5-78d8a45fdb99 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.564808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 366/746] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:27.565909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674c54-8852-406f-89a5-78d8a45fdb99 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.570827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:27.592558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d392315-55bf-42bd-9ec1-96bcf787d08a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.613357 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:27.630406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bd956-d700-46f0-9fe9-dbc8dccdb675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.631167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bd956-d700-46f0-9fe9-dbc8dccdb675 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.631449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bd956-d700-46f0-9fe9-dbc8dccdb675 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.631719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bd956-d700-46f0-9fe9-dbc8dccdb675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.643048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bd956-d700-46f0-9fe9-dbc8dccdb675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.644563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bd956-d700-46f0-9fe9-dbc8dccdb675 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-8d1bd956-d700-46f0-9fe9-dbc8dccdb675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:27.658897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:27.659932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:27.660856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:27.660856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:27.660856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:27.660856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 381/747] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:27.660856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bd956-d700-46f0-9fe9-dbc8dccdb675 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.680869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674c54-8852-406f-89a5-78d8a45fdb99 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.717434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584eeaf2-0c4d-4a1e-9e9d-a5693f02b460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.718003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584eeaf2-0c4d-4a1e-9e9d-a5693f02b460 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.718274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584eeaf2-0c4d-4a1e-9e9d-a5693f02b460 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.718536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584eeaf2-0c4d-4a1e-9e9d-a5693f02b460 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.719176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584eeaf2-0c4d-4a1e-9e9d-a5693f02b460 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.720564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584eeaf2-0c4d-4a1e-9e9d-a5693f02b460 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.740725 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 367/748] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:27 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:27.740725 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584eeaf2-0c4d-4a1e-9e9d-a5693f02b460 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.786232 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bd956-d700-46f0-9fe9-dbc8dccdb675 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.829075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:27.844667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca93f8-ab86-46be-b111-067c52dbc141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.845727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca93f8-ab86-46be-b111-067c52dbc141 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.846209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca93f8-ab86-46be-b111-067c52dbc141 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.846679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca93f8-ab86-46be-b111-067c52dbc141 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.847680 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca93f8-ab86-46be-b111-067c52dbc141 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:27.848301 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca93f8-ab86-46be-b111-067c52dbc141 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.850287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca93f8-ab86-46be-b111-067c52dbc141 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.877251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 382/749] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:27.878592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca93f8-ab86-46be-b111-067c52dbc141 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.892282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:27.897980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584eeaf2-0c4d-4a1e-9e9d-a5693f02b460 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.938065 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c80952-2daa-46d6-b207-5649a920b54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.938654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c80952-2daa-46d6-b207-5649a920b54d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.938932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c80952-2daa-46d6-b207-5649a920b54d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.939199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c80952-2daa-46d6-b207-5649a920b54d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.947147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c80952-2daa-46d6-b207-5649a920b54d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.949619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c80952-2daa-46d6-b207-5649a920b54d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c3c80952-2daa-46d6-b207-5649a920b54d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:27.968709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:27.969849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:27.970986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:27.970986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:27.970986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:27.970986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:27.970986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:27.970986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:27.970986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:27.970986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:27.972836 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 368/750] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:27.973731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c80952-2daa-46d6-b207-5649a920b54d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:28.019212 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca93f8-ab86-46be-b111-067c52dbc141 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.055513 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4dbd1-d402-41ab-b3f0-910bd500f6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.056280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4dbd1-d402-41ab-b3f0-910bd500f6ce None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.056668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4dbd1-d402-41ab-b3f0-910bd500f6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.057053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4dbd1-d402-41ab-b3f0-910bd500f6ce None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.058167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4dbd1-d402-41ab-b3f0-910bd500f6ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:28.058820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4dbd1-d402-41ab-b3f0-910bd500f6ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:28.060309 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4dbd1-d402-41ab-b3f0-910bd500f6ce None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:28.072462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 383/751] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:28.073549 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4dbd1-d402-41ab-b3f0-910bd500f6ce None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:28.095134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c80952-2daa-46d6-b207-5649a920b54d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.132119 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:28.135643 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2408a8-2438-45bb-9585-c5c82300e4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.136542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2408a8-2438-45bb-9585-c5c82300e4db None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.137281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2408a8-2438-45bb-9585-c5c82300e4db None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.138303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2408a8-2438-45bb-9585-c5c82300e4db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.151332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2408a8-2438-45bb-9585-c5c82300e4db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:28.153637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2408a8-2438-45bb-9585-c5c82300e4db None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-9f2408a8-2438-45bb-9585-c5c82300e4db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:28.173279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:28.174484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:28.175576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:28.175576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:28.175576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:28.177316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 369/752] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:28.178237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2408a8-2438-45bb-9585-c5c82300e4db None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:28.196941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4dbd1-d402-41ab-b3f0-910bd500f6ce None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.199281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:28.239214 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257afc6-c1a6-4a35-9fd5-1ff313bfec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.239982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257afc6-c1a6-4a35-9fd5-1ff313bfec28 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.240361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257afc6-c1a6-4a35-9fd5-1ff313bfec28 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.240762 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257afc6-c1a6-4a35-9fd5-1ff313bfec28 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.241558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257afc6-c1a6-4a35-9fd5-1ff313bfec28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:28.242051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257afc6-c1a6-4a35-9fd5-1ff313bfec28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:28.243673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257afc6-c1a6-4a35-9fd5-1ff313bfec28 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:28.263335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 384/753] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:28.263335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257afc6-c1a6-4a35-9fd5-1ff313bfec28 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:28.307013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2408a8-2438-45bb-9585-c5c82300e4db None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.344668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958deb04-1e3e-4b14-a1f8-6caf85c62901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.345389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958deb04-1e3e-4b14-a1f8-6caf85c62901 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.345615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958deb04-1e3e-4b14-a1f8-6caf85c62901 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.345888 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958deb04-1e3e-4b14-a1f8-6caf85c62901 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.353979 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958deb04-1e3e-4b14-a1f8-6caf85c62901 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:28.355778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958deb04-1e3e-4b14-a1f8-6caf85c62901 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-958deb04-1e3e-4b14-a1f8-6caf85c62901 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:28.367148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:28.368264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:28.369700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:28.369700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:28.369700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:28.369700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:28.369700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:28.369700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:28.369700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:28.369700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:28.369700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:28.369700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:28.373036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 370/754] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 12:20:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:28.373693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958deb04-1e3e-4b14-a1f8-6caf85c62901 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:28.376459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257afc6-c1a6-4a35-9fd5-1ff313bfec28 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.385735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:28.413247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e57b8-2646-4604-9069-adb866826d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.413839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e57b8-2646-4604-9069-adb866826d77 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.414117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e57b8-2646-4604-9069-adb866826d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.414387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e57b8-2646-4604-9069-adb866826d77 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.415041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e57b8-2646-4604-9069-adb866826d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:28.415343 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e57b8-2646-4604-9069-adb866826d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:28.416750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e57b8-2646-4604-9069-adb866826d77 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:28.429225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 385/755] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:28.430458 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af124a-3143-45fc-932e-0fc2a7107b55 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.430723 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af124a-3143-45fc-932e-0fc2a7107b55 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.431017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af124a-3143-45fc-932e-0fc2a7107b55 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.436827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 386/756] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:28.436827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af124a-3143-45fc-932e-0fc2a7107b55 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:28.458610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:28.490674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958deb04-1e3e-4b14-a1f8-6caf85c62901 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.525437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a128b3-d876-4420-9eed-95f6f12f8af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.526148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a128b3-d876-4420-9eed-95f6f12f8af7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.526431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a128b3-d876-4420-9eed-95f6f12f8af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.526788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a128b3-d876-4420-9eed-95f6f12f8af7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.527539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a128b3-d876-4420-9eed-95f6f12f8af7 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=bfec040b09c9415b9e89cd28c67aa5df) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:28.527846 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a128b3-d876-4420-9eed-95f6f12f8af7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:28.529451 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a128b3-d876-4420-9eed-95f6f12f8af7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:28.545073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 371/757] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:28 2022] GET /identity/v3/role_assignments?user.id=bfec040b09c9415b9e89cd28c67aa5df&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:28.545893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a128b3-d876-4420-9eed-95f6f12f8af7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:28.546077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af124a-3143-45fc-932e-0fc2a7107b55 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.586766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a767e-0571-44e6-a4f7-08262ac20987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.587466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a767e-0571-44e6-a4f7-08262ac20987 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.587743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a767e-0571-44e6-a4f7-08262ac20987 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.588413 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a767e-0571-44e6-a4f7-08262ac20987 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.589821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a767e-0571-44e6-a4f7-08262ac20987 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:28.590374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a767e-0571-44e6-a4f7-08262ac20987 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:28.592073 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a767e-0571-44e6-a4f7-08262ac20987 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:28.613453 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 387/758] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 24 12:20:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:28.614672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeba93-b598-4cb1-9546-40e127e9506c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.614961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeba93-b598-4cb1-9546-40e127e9506c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.615508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeba93-b598-4cb1-9546-40e127e9506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.635048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-acdeba93-b598-4cb1-9546-40e127e9506c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:28.636577 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-acdeba93-b598-4cb1-9546-40e127e9506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:28.666434 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:28.688696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a128b3-d876-4420-9eed-95f6f12f8af7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.726519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 388/759] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:28.733645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdeba93-b598-4cb1-9546-40e127e9506c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:28.735926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a114c42-645b-46a4-addf-45de09752193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.740716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114c42-645b-46a4-addf-45de09752193 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.740716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114c42-645b-46a4-addf-45de09752193 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.740716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114c42-645b-46a4-addf-45de09752193 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.740716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:28.754371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a114c42-645b-46a4-addf-45de09752193 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:28.755857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a114c42-645b-46a4-addf-45de09752193 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3a114c42-645b-46a4-addf-45de09752193 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:28.778201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:28.779147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:28.780018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:28.780018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:28.780018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:28.780018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:28.780018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:28.780018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:28.780018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:28.780018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:28.787756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 372/760] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:28.789341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a114c42-645b-46a4-addf-45de09752193 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:28.944957 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdeba93-b598-4cb1-9546-40e127e9506c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.983194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecc546-84bf-4fa0-9079-9fea225fe357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.983910 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecc546-84bf-4fa0-9079-9fea225fe357 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.984192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecc546-84bf-4fa0-9079-9fea225fe357 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.984826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecc546-84bf-4fa0-9079-9fea225fe357 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.986609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a114c42-645b-46a4-addf-45de09752193 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.025609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:29.051256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc71a1-7462-459e-a26e-f589812adb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:29.052406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc71a1-7462-459e-a26e-f589812adb85 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.052951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc71a1-7462-459e-a26e-f589812adb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.053629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc71a1-7462-459e-a26e-f589812adb85 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.054893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc71a1-7462-459e-a26e-f589812adb85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:29.055371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc71a1-7462-459e-a26e-f589812adb85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:29.057880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc71a1-7462-459e-a26e-f589812adb85 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:29.074396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ecc546-84bf-4fa0-9079-9fea225fe357 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:29.075956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ecc546-84bf-4fa0-9079-9fea225fe357 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:29.076172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 373/761] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:29.077310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc71a1-7462-459e-a26e-f589812adb85 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.160312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:29.185226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 389/762] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:20:29.186839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecc546-84bf-4fa0-9079-9fea225fe357 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.211654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc71a1-7462-459e-a26e-f589812adb85 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.247830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cf787-0d68-41f0-bd87-731782e3894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:29.248482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cf787-0d68-41f0-bd87-731782e3894f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.249010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cf787-0d68-41f0-bd87-731782e3894f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.249349 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cf787-0d68-41f0-bd87-731782e3894f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.249997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cf787-0d68-41f0-bd87-731782e3894f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:29.251553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cf787-0d68-41f0-bd87-731782e3894f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:29.258769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-208cf787-0d68-41f0-bd87-731782e3894f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:29.261475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:29.262900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:29.269210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 374/763] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:29.269981 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cf787-0d68-41f0-bd87-731782e3894f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.316796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ecc546-84bf-4fa0-9079-9fea225fe357 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.356175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb82e16-cd38-449f-b896-2ce3895a0c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:29.356810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb82e16-cd38-449f-b896-2ce3895a0c91 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.357086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb82e16-cd38-449f-b896-2ce3895a0c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.357354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb82e16-cd38-449f-b896-2ce3895a0c91 None admin] PATH_INFO: `/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/e65026e01df242e7be11ed24214a4efa/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.359552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb82e16-cd38-449f-b896-2ce3895a0c91 None admin] RBAC: Authorizing `identity:create_grant(project_id=70a5f9cf0499405d82f5f5a47c2877eb, user_id=e65026e01df242e7be11ed24214a4efa, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:29.360990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb82e16-cd38-449f-b896-2ce3895a0c91 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:29.381182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 390/764] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:29 2022] PUT /identity/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/e65026e01df242e7be11ed24214a4efa/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:29.382236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb82e16-cd38-449f-b896-2ce3895a0c91 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.404836 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cf787-0d68-41f0-bd87-731782e3894f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.455574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:29.496328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed1565-05a8-4b81-b11d-307a83a4742c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:29.497292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed1565-05a8-4b81-b11d-307a83a4742c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.497426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed1565-05a8-4b81-b11d-307a83a4742c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.497770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed1565-05a8-4b81-b11d-307a83a4742c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.520705 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb82e16-cd38-449f-b896-2ce3895a0c91 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.522175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed1565-05a8-4b81-b11d-307a83a4742c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:29.523903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed1565-05a8-4b81-b11d-307a83a4742c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:29.546303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c5ed1565-05a8-4b81-b11d-307a83a4742c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:29.551115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:29.552227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:29.553369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:29.553369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:29.553369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:29.553369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 375/765] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:29.562618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed1565-05a8-4b81-b11d-307a83a4742c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.570253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dd643-56ae-4d6e-87ed-c696a1809bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:29.571205 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dd643-56ae-4d6e-87ed-c696a1809bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.571487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dd643-56ae-4d6e-87ed-c696a1809bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.572852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dd643-56ae-4d6e-87ed-c696a1809bd6 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.573836 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dd643-56ae-4d6e-87ed-c696a1809bd6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:29.574271 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dd643-56ae-4d6e-87ed-c696a1809bd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:29.575703 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dd643-56ae-4d6e-87ed-c696a1809bd6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:29.581587 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 391/766] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:29.590238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dd643-56ae-4d6e-87ed-c696a1809bd6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.745009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed1565-05a8-4b81-b11d-307a83a4742c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.747123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dd643-56ae-4d6e-87ed-c696a1809bd6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.786603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd10653-baf9-4c97-b3c1-4734d70dda51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:29.787224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd10653-baf9-4c97-b3c1-4734d70dda51 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.787505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd10653-baf9-4c97-b3c1-4734d70dda51 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.787771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd10653-baf9-4c97-b3c1-4734d70dda51 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.788411 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd10653-baf9-4c97-b3c1-4734d70dda51 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:29.788735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd10653-baf9-4c97-b3c1-4734d70dda51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:29.790791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd10653-baf9-4c97-b3c1-4734d70dda51 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:29.803455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 392/767] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:29.805026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd10653-baf9-4c97-b3c1-4734d70dda51 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.816519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffface3-2367-457d-b606-1fedd857e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:29.817165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffface3-2367-457d-b606-1fedd857e66d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.817552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffface3-2367-457d-b606-1fedd857e66d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.817826 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffface3-2367-457d-b606-1fedd857e66d None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.818478 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffface3-2367-457d-b606-1fedd857e66d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:29.818776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffface3-2367-457d-b606-1fedd857e66d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:29.820277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffface3-2367-457d-b606-1fedd857e66d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:29.839089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 376/768] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:29.840546 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae9b7-79bf-413b-a4a7-afb86ff2039e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.841900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae9b7-79bf-413b-a4a7-afb86ff2039e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.845072 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae9b7-79bf-413b-a4a7-afb86ff2039e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.846809 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 377/769] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:29.847614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae9b7-79bf-413b-a4a7-afb86ff2039e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.883078 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:29.887555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:29.941789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd10653-baf9-4c97-b3c1-4734d70dda51 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.988467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae9b7-79bf-413b-a4a7-afb86ff2039e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.992537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cba15-83c5-4e86-b6f4-cdb5d2bdb307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:29.993188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cba15-83c5-4e86-b6f4-cdb5d2bdb307 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.993466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cba15-83c5-4e86-b6f4-cdb5d2bdb307 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.993735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cba15-83c5-4e86-b6f4-cdb5d2bdb307 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.994442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cba15-83c5-4e86-b6f4-cdb5d2bdb307 None admin] RBAC: Adding query filter params (interface=public, service_id=96ba078b233947a6960604e0fa863e5f, region_id=RegionOne) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:29.994745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cba15-83c5-4e86-b6f4-cdb5d2bdb307 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:29.998298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cba15-83c5-4e86-b6f4-cdb5d2bdb307 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.009650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 393/770] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/endpoints?service_id=96ba078b233947a6960604e0fa863e5f&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:30.011811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75d920-8490-448d-bded-47e214115c77 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.012292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75d920-8490-448d-bded-47e214115c77 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.012785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75d920-8490-448d-bded-47e214115c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.029794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217d5c3-38db-479b-a4c5-38b0f377b9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.030451 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d5c3-38db-479b-a4c5-38b0f377b9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.031050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d5c3-38db-479b-a4c5-38b0f377b9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.031050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d5c3-38db-479b-a4c5-38b0f377b9c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.037067 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3f75d920-8490-448d-bded-47e214115c77 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:30.038609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217d5c3-38db-479b-a4c5-38b0f377b9c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.039157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3f75d920-8490-448d-bded-47e214115c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:30.040358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217d5c3-38db-479b-a4c5-38b0f377b9c5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-6217d5c3-38db-479b-a4c5-38b0f377b9c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:30.052623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:30.053712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:30.054851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:30.054851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:30.054851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:30.054851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:30.054851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:30.054851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:30.054851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:30.054851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:30.056347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 378/771] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:30.061542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217d5c3-38db-479b-a4c5-38b0f377b9c5 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.133070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 394/772] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:30.134189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75d920-8490-448d-bded-47e214115c77 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.182025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:30.205958 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:30.231076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217d5c3-38db-479b-a4c5-38b0f377b9c5 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.262695 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75d920-8490-448d-bded-47e214115c77 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.303560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524e5cd-0d64-479b-b8ec-069a7d768e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.304610 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524e5cd-0d64-479b-b8ec-069a7d768e17 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.305052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524e5cd-0d64-479b-b8ec-069a7d768e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.305674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524e5cd-0d64-479b-b8ec-069a7d768e17 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.306628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524e5cd-0d64-479b-b8ec-069a7d768e17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:30.307076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524e5cd-0d64-479b-b8ec-069a7d768e17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.308986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524e5cd-0d64-479b-b8ec-069a7d768e17 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.310371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323d137-ef22-43af-a04c-168e870115a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.311197 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323d137-ef22-43af-a04c-168e870115a9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.311375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323d137-ef22-43af-a04c-168e870115a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.311765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323d137-ef22-43af-a04c-168e870115a9 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.312450 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323d137-ef22-43af-a04c-168e870115a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.313978 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323d137-ef22-43af-a04c-168e870115a9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.324000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 395/773] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:30.327012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 379/774] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/services => generated 772 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:30.336281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524e5cd-0d64-479b-b8ec-069a7d768e17 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.480523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:30.485742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524e5cd-0d64-479b-b8ec-069a7d768e17 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.498591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323d137-ef22-43af-a04c-168e870115a9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.530264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f1a3a-ffe1-4c42-aedd-5efcebef9369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.531593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f1a3a-ffe1-4c42-aedd-5efcebef9369 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.532056 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f1a3a-ffe1-4c42-aedd-5efcebef9369 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.532785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f1a3a-ffe1-4c42-aedd-5efcebef9369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.533960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f1a3a-ffe1-4c42-aedd-5efcebef9369 None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=34a3cadee8ea4b3e9dff0bc23dfe9dc5) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:30.534464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f1a3a-ffe1-4c42-aedd-5efcebef9369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.536732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f1a3a-ffe1-4c42-aedd-5efcebef9369 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.544669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 396/775] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/role_assignments?user.id=34a3cadee8ea4b3e9dff0bc23dfe9dc5&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:30.625168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323d137-ef22-43af-a04c-168e870115a9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.664188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9dd97-2340-4199-a7c5-24cb169c2283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.664874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9dd97-2340-4199-a7c5-24cb169c2283 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.665151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9dd97-2340-4199-a7c5-24cb169c2283 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.665419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9dd97-2340-4199-a7c5-24cb169c2283 None admin] PATH_INFO: `/v3/services/network` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.666018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9dd97-2340-4199-a7c5-24cb169c2283 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.667499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9dd97-2340-4199-a7c5-24cb169c2283 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f0a9dd97-2340-4199-a7c5-24cb169c2283 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:30.677415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:30.678502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:30.679344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 380/776] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:30.684854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f1a3a-ffe1-4c42-aedd-5efcebef9369 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.707329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608db1b0-a8c1-4f9c-b92e-a265f00f0f44 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.707696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608db1b0-a8c1-4f9c-b92e-a265f00f0f44 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.707886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608db1b0-a8c1-4f9c-b92e-a265f00f0f44 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.709819 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 381/777] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:30.716036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0f615-2cae-4893-8ba8-5b53421710ce None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.716412 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0f615-2cae-4893-8ba8-5b53421710ce None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.716576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0f615-2cae-4893-8ba8-5b53421710ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.745834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ced0f615-2cae-4893-8ba8-5b53421710ce None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:30.747080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ced0f615-2cae-4893-8ba8-5b53421710ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:30.765770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:30.827075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f1a3a-ffe1-4c42-aedd-5efcebef9369 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.867167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 382/778] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:30.868892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1e431-fd85-42be-a30e-360215595b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.869734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1e431-fd85-42be-a30e-360215595b8f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.870293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1e431-fd85-42be-a30e-360215595b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.870839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1e431-fd85-42be-a30e-360215595b8f None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.871696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1e431-fd85-42be-a30e-360215595b8f None admin] RBAC: Adding query filter params (name=network) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:30.872193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1e431-fd85-42be-a30e-360215595b8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.873809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1e431-fd85-42be-a30e-360215595b8f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.879162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 397/779] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:30.884589 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0f615-2cae-4893-8ba8-5b53421710ce None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.057077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0f615-2cae-4893-8ba8-5b53421710ce None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.110793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe64280-fcbf-4770-817d-12c2066f74f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.111413 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe64280-fcbf-4770-817d-12c2066f74f0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.111688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe64280-fcbf-4770-817d-12c2066f74f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.111953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe64280-fcbf-4770-817d-12c2066f74f0 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.112594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe64280-fcbf-4770-817d-12c2066f74f0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:31.112937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe64280-fcbf-4770-817d-12c2066f74f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.114320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe64280-fcbf-4770-817d-12c2066f74f0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.125621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 383/780] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:31.131070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1e431-fd85-42be-a30e-360215595b8f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.175794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:31.258012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1e431-fd85-42be-a30e-360215595b8f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.293917 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b72efa-dafb-4985-8da7-b1d743cd3b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.294896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b72efa-dafb-4985-8da7-b1d743cd3b67 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.295405 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b72efa-dafb-4985-8da7-b1d743cd3b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.295510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe64280-fcbf-4770-817d-12c2066f74f0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.296372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b72efa-dafb-4985-8da7-b1d743cd3b67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.297313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b72efa-dafb-4985-8da7-b1d743cd3b67 None admin] RBAC: Adding query filter params (interface=public, service_id=adcff934dcf346998b14c0c3cef55f91, region_id=RegionOne) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:31.297816 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b72efa-dafb-4985-8da7-b1d743cd3b67 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.299414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b72efa-dafb-4985-8da7-b1d743cd3b67 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.305427 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 398/781] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/endpoints?service_id=adcff934dcf346998b14c0c3cef55f91&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:31.310863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b72efa-dafb-4985-8da7-b1d743cd3b67 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.347141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:31.409336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:31.420561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b72efa-dafb-4985-8da7-b1d743cd3b67 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.456176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d70a95-3cfa-4750-907b-07af2640b3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.456811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d70a95-3cfa-4750-907b-07af2640b3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.457089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d70a95-3cfa-4750-907b-07af2640b3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.457362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d70a95-3cfa-4750-907b-07af2640b3b8 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.457972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d70a95-3cfa-4750-907b-07af2640b3b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.459344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d70a95-3cfa-4750-907b-07af2640b3b8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.465357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 399/782] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/services => generated 772 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:31.534853 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe64280-fcbf-4770-817d-12c2066f74f0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.582570 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b5c5b-ceb4-455c-bb3e-9c8e9ef43a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.583203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b5c5b-ceb4-455c-bb3e-9c8e9ef43a6a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.583472 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b5c5b-ceb4-455c-bb3e-9c8e9ef43a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.583736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b5c5b-ceb4-455c-bb3e-9c8e9ef43a6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.596161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b5c5b-ceb4-455c-bb3e-9c8e9ef43a6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f48b5c5b-ceb4-455c-bb3e-9c8e9ef43a6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:31.600817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:31.601749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:31.602561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:31.602561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:31.602561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:31.602561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:31.602561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:31.602561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:31.602561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:31.602561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:31.602561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:31.602561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:31.602561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:31.602561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:31.603197 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 384/783] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:31.610113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d70a95-3cfa-4750-907b-07af2640b3b8 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.747853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d70a95-3cfa-4750-907b-07af2640b3b8 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.765126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6935c4b-d654-49fe-9987-3b32813e9d30 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.765400 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6935c4b-d654-49fe-9987-3b32813e9d30 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.765808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6935c4b-d654-49fe-9987-3b32813e9d30 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.767346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 385/784] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:31.772795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc77c8c-12e6-4290-852a-5230a4a7ad0a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.773077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc77c8c-12e6-4290-852a-5230a4a7ad0a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.773348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc77c8c-12e6-4290-852a-5230a4a7ad0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.788625 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51390823-0204-4b11-a818-8039f3c8aa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.789264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51390823-0204-4b11-a818-8039f3c8aa0c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.789539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51390823-0204-4b11-a818-8039f3c8aa0c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.789800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51390823-0204-4b11-a818-8039f3c8aa0c None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.790472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51390823-0204-4b11-a818-8039f3c8aa0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:31.790771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51390823-0204-4b11-a818-8039f3c8aa0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.792197 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51390823-0204-4b11-a818-8039f3c8aa0c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.799421 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-abc77c8c-12e6-4290-852a-5230a4a7ad0a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:31.801550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-abc77c8c-12e6-4290-852a-5230a4a7ad0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:31.804515 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 400/785] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:31.810075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51390823-0204-4b11-a818-8039f3c8aa0c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.833335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:31.906609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 386/786] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:31.931468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51390823-0204-4b11-a818-8039f3c8aa0c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.969155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc12f0-a4ee-49bd-82b1-23ed364f050d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.970142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc12f0-a4ee-49bd-82b1-23ed364f050d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.970650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc12f0-a4ee-49bd-82b1-23ed364f050d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.971138 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc12f0-a4ee-49bd-82b1-23ed364f050d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.983538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc12f0-a4ee-49bd-82b1-23ed364f050d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.985390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc12f0-a4ee-49bd-82b1-23ed364f050d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-8ecc12f0-a4ee-49bd-82b1-23ed364f050d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:32.000744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:32.001659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:32.002554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:32.002554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:32.002554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:32.002554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 401/787] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:32.009751 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc77c8c-12e6-4290-852a-5230a4a7ad0a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.016012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:32.123582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc77c8c-12e6-4290-852a-5230a4a7ad0a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.164458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa7754-f11c-4070-b0aa-7e02beb448fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:32.165109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa7754-f11c-4070-b0aa-7e02beb448fe None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.165393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa7754-f11c-4070-b0aa-7e02beb448fe None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.165726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa7754-f11c-4070-b0aa-7e02beb448fe None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.166390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa7754-f11c-4070-b0aa-7e02beb448fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:32.166699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa7754-f11c-4070-b0aa-7e02beb448fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:32.168837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa7754-f11c-4070-b0aa-7e02beb448fe None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:32.181712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 387/788] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:32.189787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc12f0-a4ee-49bd-82b1-23ed364f050d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.200752 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa7754-f11c-4070-b0aa-7e02beb448fe None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.321119 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:32.326767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:32.386074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc12f0-a4ee-49bd-82b1-23ed364f050d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.392502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa7754-f11c-4070-b0aa-7e02beb448fe None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.430402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35dd1eb-cca0-464e-9c2a-f0343a10ceba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:32.431102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dd1eb-cca0-464e-9c2a-f0343a10ceba None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.431447 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dd1eb-cca0-464e-9c2a-f0343a10ceba None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.431723 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dd1eb-cca0-464e-9c2a-f0343a10ceba None admin] PATH_INFO: `/v3/users/swift` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.440788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35dd1eb-cca0-464e-9c2a-f0343a10ceba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:32.442314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35dd1eb-cca0-464e-9c2a-f0343a10ceba None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-f35dd1eb-cca0-464e-9c2a-f0343a10ceba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:32.455167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:32.456334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:32.457538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:32.457538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:32.457538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:32.457538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:32.457538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:32.457538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:32.457538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:32.457538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:32.457538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:32.457538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:32.457538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 402/789] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:32.463553 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35dd1eb-cca0-464e-9c2a-f0343a10ceba None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.514171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea07be9-6aa2-4378-95eb-3e937e3f4dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:32.514818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea07be9-6aa2-4378-95eb-3e937e3f4dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.515292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea07be9-6aa2-4378-95eb-3e937e3f4dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.515737 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea07be9-6aa2-4378-95eb-3e937e3f4dc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.553845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea07be9-6aa2-4378-95eb-3e937e3f4dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-6ea07be9-6aa2-4378-95eb-3e937e3f4dc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:32.558687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:32.559610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:32.560519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:32.560519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:32.560519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:32.560519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:32.560519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:32.560519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:32.560519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:32.560519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:32.560519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:32.560519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:32.560519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:32.560519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:32.572883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 388/790] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:32.573737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35dd1eb-cca0-464e-9c2a-f0343a10ceba None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.574986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea07be9-6aa2-4378-95eb-3e937e3f4dc6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.607114 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:32.613341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59954a88-54db-4cd1-aba3-25c010ae4937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:32.613877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59954a88-54db-4cd1-aba3-25c010ae4937 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.614735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59954a88-54db-4cd1-aba3-25c010ae4937 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.615011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59954a88-54db-4cd1-aba3-25c010ae4937 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.615819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59954a88-54db-4cd1-aba3-25c010ae4937 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:32.616126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59954a88-54db-4cd1-aba3-25c010ae4937 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:32.617821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59954a88-54db-4cd1-aba3-25c010ae4937 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:32.638476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 403/791] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 24 12:20:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:32.643507 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59954a88-54db-4cd1-aba3-25c010ae4937 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.752886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea07be9-6aa2-4378-95eb-3e937e3f4dc6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.781895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:32.793723 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ade2ff-6450-4ca1-af08-734066c40a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:32.794400 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ade2ff-6450-4ca1-af08-734066c40a5a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.794711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ade2ff-6450-4ca1-af08-734066c40a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.794943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ade2ff-6450-4ca1-af08-734066c40a5a None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.796483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ade2ff-6450-4ca1-af08-734066c40a5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:32.796829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ade2ff-6450-4ca1-af08-734066c40a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:32.798234 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ade2ff-6450-4ca1-af08-734066c40a5a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:32.808568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 389/792] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:32.813837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ade2ff-6450-4ca1-af08-734066c40a5a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.839830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59954a88-54db-4cd1-aba3-25c010ae4937 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.880236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86437cd-e6dc-49b3-b38f-bf306f785476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:32.880994 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86437cd-e6dc-49b3-b38f-bf306f785476 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.881368 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86437cd-e6dc-49b3-b38f-bf306f785476 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.881642 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86437cd-e6dc-49b3-b38f-bf306f785476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.893208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86437cd-e6dc-49b3-b38f-bf306f785476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:32.894679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86437cd-e6dc-49b3-b38f-bf306f785476 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-c86437cd-e6dc-49b3-b38f-bf306f785476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:32.911121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:32.912117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:32.913017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:32.913017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:32.913017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:32.913017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 404/793] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:32.914047 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76629927-ffdb-4dfe-9d54-3c8d769f8d41 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.914339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76629927-ffdb-4dfe-9d54-3c8d769f8d41 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.914620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76629927-ffdb-4dfe-9d54-3c8d769f8d41 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.916091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 405/794] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:32.917396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cea25-1f5f-4b1a-bef7-542287f240dd None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.917676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cea25-1f5f-4b1a-bef7-542287f240dd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.917948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cea25-1f5f-4b1a-bef7-542287f240dd None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.919239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 406/795] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:32.920260 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cea25-1f5f-4b1a-bef7-542287f240dd None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.964635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ade2ff-6450-4ca1-af08-734066c40a5a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.966577 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:33.008531 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205d0f6-2d2a-409f-b5d5-4342fbff45fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:33.009194 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205d0f6-2d2a-409f-b5d5-4342fbff45fb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.009470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205d0f6-2d2a-409f-b5d5-4342fbff45fb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.009739 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205d0f6-2d2a-409f-b5d5-4342fbff45fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.021060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205d0f6-2d2a-409f-b5d5-4342fbff45fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:33.022541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205d0f6-2d2a-409f-b5d5-4342fbff45fb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b205d0f6-2d2a-409f-b5d5-4342fbff45fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:33.036880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:33.037837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:33.038715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:33.038715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:33.038715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:33.038885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cea25-1f5f-4b1a-bef7-542287f240dd None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:33.042914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 390/796] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:33.044237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016ab55-8733-418f-b7a5-b4e2dd8523ea None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.044477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016ab55-8733-418f-b7a5-b4e2dd8523ea None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.044794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016ab55-8733-418f-b7a5-b4e2dd8523ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.064207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6016ab55-8733-418f-b7a5-b4e2dd8523ea None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:33.065498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6016ab55-8733-418f-b7a5-b4e2dd8523ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:33.078596 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5098f3-e18e-4960-8966-d2b72ace832b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:33.079201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5098f3-e18e-4960-8966-d2b72ace832b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.079491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5098f3-e18e-4960-8966-d2b72ace832b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.079740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5098f3-e18e-4960-8966-d2b72ace832b None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.080503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5098f3-e18e-4960-8966-d2b72ace832b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:33.080711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5098f3-e18e-4960-8966-d2b72ace832b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:33.082057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5098f3-e18e-4960-8966-d2b72ace832b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:33.098893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 407/797] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:33.098893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37b99c-ab51-4834-82c8-8cb72f696f56 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.098893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37b99c-ab51-4834-82c8-8cb72f696f56 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.098893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37b99c-ab51-4834-82c8-8cb72f696f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.103334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:33.114893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ce37b99c-ab51-4834-82c8-8cb72f696f56 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:33.116107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ce37b99c-ab51-4834-82c8-8cb72f696f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:33.147012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 391/798] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:33.147867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016ab55-8733-418f-b7a5-b4e2dd8523ea None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:33.216986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 408/799] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:33.218516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37b99c-ab51-4834-82c8-8cb72f696f56 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:33.251156 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:33.310583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016ab55-8733-418f-b7a5-b4e2dd8523ea None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:33.345360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37b99c-ab51-4834-82c8-8cb72f696f56 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:33.386222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657bb3b-97e8-4627-b378-cbd7ff22c50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:33.387179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657bb3b-97e8-4627-b378-cbd7ff22c50e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.387633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657bb3b-97e8-4627-b378-cbd7ff22c50e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.388094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657bb3b-97e8-4627-b378-cbd7ff22c50e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.393394 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150a005-b72d-4b87-99f7-4d162f8e6e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:33.394084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150a005-b72d-4b87-99f7-4d162f8e6e0b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.394366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150a005-b72d-4b87-99f7-4d162f8e6e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.394640 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150a005-b72d-4b87-99f7-4d162f8e6e0b None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.395347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150a005-b72d-4b87-99f7-4d162f8e6e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:33.395653 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150a005-b72d-4b87-99f7-4d162f8e6e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:33.397193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150a005-b72d-4b87-99f7-4d162f8e6e0b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:33.398057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657bb3b-97e8-4627-b378-cbd7ff22c50e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:33.399971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657bb3b-97e8-4627-b378-cbd7ff22c50e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-d657bb3b-97e8-4627-b378-cbd7ff22c50e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:33.412873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:33.413807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:33.414732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:33.414732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:33.414732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:33.414732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:33.414732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:33.414732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:33.414732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:33.414732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:33.417461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 409/800] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:33.424660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 392/801] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:33.426016 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657bb3b-97e8-4627-b378-cbd7ff22c50e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:33.426980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150a005-b72d-4b87-99f7-4d162f8e6e0b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:33.459211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:33.540258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:33.549833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150a005-b72d-4b87-99f7-4d162f8e6e0b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:33.557090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657bb3b-97e8-4627-b378-cbd7ff22c50e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:33.592670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47632e2-94b5-4ba9-b106-26e5a99d7628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:33.593324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47632e2-94b5-4ba9-b106-26e5a99d7628 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.593590 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47632e2-94b5-4ba9-b106-26e5a99d7628 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.593855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47632e2-94b5-4ba9-b106-26e5a99d7628 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.594437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47632e2-94b5-4ba9-b106-26e5a99d7628 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:33.595279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47169659-2e1a-42c8-b012-1e3cae919a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:33.595818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47632e2-94b5-4ba9-b106-26e5a99d7628 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:33.595918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47169659-2e1a-42c8-b012-1e3cae919a48 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.596191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47169659-2e1a-42c8-b012-1e3cae919a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.596452 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47169659-2e1a-42c8-b012-1e3cae919a48 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.597135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47169659-2e1a-42c8-b012-1e3cae919a48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:33.597443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47169659-2e1a-42c8-b012-1e3cae919a48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:33.598907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47169659-2e1a-42c8-b012-1e3cae919a48 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-a47632e2-94b5-4ba9-b106-26e5a99d7628 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:33.605792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:33.606765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:33.607586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 393/802] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Aug 24 12:20:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:33.608391 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47632e2-94b5-4ba9-b106-26e5a99d7628 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:33.620014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 410/803] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:33.620014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47169659-2e1a-42c8-b012-1e3cae919a48 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:33.711240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:33.720942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47632e2-94b5-4ba9-b106-26e5a99d7628 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:33.725922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47169659-2e1a-42c8-b012-1e3cae919a48 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:33.757373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713d493-77f7-4f7d-a793-0e2069ed32a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:33.758003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713d493-77f7-4f7d-a793-0e2069ed32a7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.758279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713d493-77f7-4f7d-a793-0e2069ed32a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.758551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713d493-77f7-4f7d-a793-0e2069ed32a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.766215 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e6a1e-560b-4f55-8d55-cd732d0998d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:33.766471 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713d493-77f7-4f7d-a793-0e2069ed32a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:33.766842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e6a1e-560b-4f55-8d55-cd732d0998d8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.767117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e6a1e-560b-4f55-8d55-cd732d0998d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.767385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e6a1e-560b-4f55-8d55-cd732d0998d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.767943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713d493-77f7-4f7d-a793-0e2069ed32a7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:33.779099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e6a1e-560b-4f55-8d55-cd732d0998d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-4713d493-77f7-4f7d-a793-0e2069ed32a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:33.779327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:33.780188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:33.781129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:33.781129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:33.781129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:33.781129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:33.781129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:33.781129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:33.781129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:33.781129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:33.781129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:33.781129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6c5e6a1e-560b-4f55-8d55-cd732d0998d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:33.783890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:33.785050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:33.789400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:33.789400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:33.789400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:33.789400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:33.789400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:33.789400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:33.789400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:33.789400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:33.789400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:33.789400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:33.789400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:33.789400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:33.790074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 394/804] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 12:20:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:33.790146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 411/805] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:33.790213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713d493-77f7-4f7d-a793-0e2069ed32a7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:33.790290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e6a1e-560b-4f55-8d55-cd732d0998d8 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:33.799480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:33.894891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e6a1e-560b-4f55-8d55-cd732d0998d8 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:33.897835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713d493-77f7-4f7d-a793-0e2069ed32a7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:33.929553 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d4b2e-7a5c-4cc2-a9c1-f7e369a4e295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:33.930213 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d4b2e-7a5c-4cc2-a9c1-f7e369a4e295 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.930497 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d4b2e-7a5c-4cc2-a9c1-f7e369a4e295 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.930767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d4b2e-7a5c-4cc2-a9c1-f7e369a4e295 None admin] PATH_INFO: `/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/bfec040b09c9415b9e89cd28c67aa5df/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.931772 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53531241-b572-4a87-9e1d-be99a732edeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:33.932374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53531241-b572-4a87-9e1d-be99a732edeb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.932727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53531241-b572-4a87-9e1d-be99a732edeb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.932874 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d4b2e-7a5c-4cc2-a9c1-f7e369a4e295 None admin] RBAC: Authorizing `identity:create_grant(project_id=70a5f9cf0499405d82f5f5a47c2877eb, user_id=bfec040b09c9415b9e89cd28c67aa5df, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:33.933012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53531241-b572-4a87-9e1d-be99a732edeb None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.933612 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53531241-b572-4a87-9e1d-be99a732edeb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:33.933901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53531241-b572-4a87-9e1d-be99a732edeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:33.934236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d4b2e-7a5c-4cc2-a9c1-f7e369a4e295 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:33.935269 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53531241-b572-4a87-9e1d-be99a732edeb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:33.939799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 395/806] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 24 12:20:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:33.940643 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53531241-b572-4a87-9e1d-be99a732edeb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:33.949635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 412/807] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:33 2022] PUT /identity/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/bfec040b09c9415b9e89cd28c67aa5df/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:33.950492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d4b2e-7a5c-4cc2-a9c1-f7e369a4e295 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.041706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:34.083494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d4b2e-7a5c-4cc2-a9c1-f7e369a4e295 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.085673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:34.127865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27115683-9be5-4889-92be-60a3474109d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.128474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27115683-9be5-4889-92be-60a3474109d2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.128784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27115683-9be5-4889-92be-60a3474109d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.129054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27115683-9be5-4889-92be-60a3474109d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.129718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27115683-9be5-4889-92be-60a3474109d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:34.130018 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27115683-9be5-4889-92be-60a3474109d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.131500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27115683-9be5-4889-92be-60a3474109d2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:34.145221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 413/808] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:34.147736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86c183-e913-43e5-ab89-8a3942bde88d None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.147736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86c183-e913-43e5-ab89-8a3942bde88d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.148075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86c183-e913-43e5-ab89-8a3942bde88d None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.150980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 414/809] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:34.152130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86c183-e913-43e5-ab89-8a3942bde88d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.181423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53531241-b572-4a87-9e1d-be99a732edeb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.219047 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d133d9-7570-40d8-b5e7-ec7f5c470d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.219664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d133d9-7570-40d8-b5e7-ec7f5c470d2d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.219952 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d133d9-7570-40d8-b5e7-ec7f5c470d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.220220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d133d9-7570-40d8-b5e7-ec7f5c470d2d None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.220988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d133d9-7570-40d8-b5e7-ec7f5c470d2d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:34.221286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d133d9-7570-40d8-b5e7-ec7f5c470d2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.222717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d133d9-7570-40d8-b5e7-ec7f5c470d2d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:34.242338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 396/810] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 12:20:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:34.242983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d133d9-7570-40d8-b5e7-ec7f5c470d2d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.269127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86c183-e913-43e5-ab89-8a3942bde88d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.299132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:34.306421 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3aba8-452c-43cf-820d-3c587c363192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.307124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3aba8-452c-43cf-820d-3c587c363192 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.307399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3aba8-452c-43cf-820d-3c587c363192 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.307699 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3aba8-452c-43cf-820d-3c587c363192 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.308331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3aba8-452c-43cf-820d-3c587c363192 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:34.308633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3aba8-452c-43cf-820d-3c587c363192 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.310060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3aba8-452c-43cf-820d-3c587c363192 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:34.319568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 415/811] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 24 12:20:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:34.319568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180b965-475b-4ca0-b5a3-6f3b7f69b452 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.319568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180b965-475b-4ca0-b5a3-6f3b7f69b452 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.319568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180b965-475b-4ca0-b5a3-6f3b7f69b452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.326864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:34.335885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7180b965-475b-4ca0-b5a3-6f3b7f69b452 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:34.337248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7180b965-475b-4ca0-b5a3-6f3b7f69b452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:34.357175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d133d9-7570-40d8-b5e7-ec7f5c470d2d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.392491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d0b2f-84f9-4d8f-bf8a-c38432950284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.393169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d0b2f-84f9-4d8f-bf8a-c38432950284 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.393442 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d0b2f-84f9-4d8f-bf8a-c38432950284 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.393983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d0b2f-84f9-4d8f-bf8a-c38432950284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.404756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d0b2f-84f9-4d8f-bf8a-c38432950284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.406649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d0b2f-84f9-4d8f-bf8a-c38432950284 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:34.417265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 416/812] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:34.418884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180b965-475b-4ca0-b5a3-6f3b7f69b452 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-001d0b2f-84f9-4d8f-bf8a-c38432950284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:34.423436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:34.424352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:34.425297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:34.425297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:34.425297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:34.429171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 397/813] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:34.429990 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d0b2f-84f9-4d8f-bf8a-c38432950284 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.536037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d0b2f-84f9-4d8f-bf8a-c38432950284 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.554881 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180b965-475b-4ca0-b5a3-6f3b7f69b452 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.562504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:34.577350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2fb76-7201-453d-9e4f-b60b54b594cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.577960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2fb76-7201-453d-9e4f-b60b54b594cb None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.578221 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2fb76-7201-453d-9e4f-b60b54b594cb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.578492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2fb76-7201-453d-9e4f-b60b54b594cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.579150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2fb76-7201-453d-9e4f-b60b54b594cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.580528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2fb76-7201-453d-9e4f-b60b54b594cb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:34.597048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:34.608753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 398/814] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:34.609917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2fb76-7201-453d-9e4f-b60b54b594cb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.615255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954d09c-6ae2-4e55-b5ae-7e4a5bce1a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.616201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954d09c-6ae2-4e55-b5ae-7e4a5bce1a1a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.616671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954d09c-6ae2-4e55-b5ae-7e4a5bce1a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.617125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954d09c-6ae2-4e55-b5ae-7e4a5bce1a1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.694556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954d09c-6ae2-4e55-b5ae-7e4a5bce1a1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.696270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954d09c-6ae2-4e55-b5ae-7e4a5bce1a1a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:34.727924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2fb76-7201-453d-9e4f-b60b54b594cb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.782361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bbe71-904c-466e-a284-af4aa4417887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.783241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bbe71-904c-466e-a284-af4aa4417887 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.783533 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bbe71-904c-466e-a284-af4aa4417887 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.783805 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bbe71-904c-466e-a284-af4aa4417887 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.784468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bbe71-904c-466e-a284-af4aa4417887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:34.784800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bbe71-904c-466e-a284-af4aa4417887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.787363 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bbe71-904c-466e-a284-af4aa4417887 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:34.803372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 399/815] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:34.804600 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bbe71-904c-466e-a284-af4aa4417887 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.822052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 417/816] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:20:34.823492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954d09c-6ae2-4e55-b5ae-7e4a5bce1a1a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.849295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:34.862175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:34.946223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bbe71-904c-466e-a284-af4aa4417887 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.951700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954d09c-6ae2-4e55-b5ae-7e4a5bce1a1a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.996591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e34441-4346-4b01-ad17-efdcd535a245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.997276 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e34441-4346-4b01-ad17-efdcd535a245 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.997561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e34441-4346-4b01-ad17-efdcd535a245 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.997832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e34441-4346-4b01-ad17-efdcd535a245 None admin] PATH_INFO: `/v3/services/network` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.998424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e34441-4346-4b01-ad17-efdcd535a245 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.999813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e34441-4346-4b01-ad17-efdcd535a245 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.000872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a07a30-68a3-4afd-8175-f9f4b1659cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.001504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a07a30-68a3-4afd-8175-f9f4b1659cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.001780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a07a30-68a3-4afd-8175-f9f4b1659cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.002049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a07a30-68a3-4afd-8175-f9f4b1659cd0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.010020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a07a30-68a3-4afd-8175-f9f4b1659cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-52e34441-4346-4b01-ad17-efdcd535a245 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:35.010543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:35.011796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:35.012666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:35.013616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a07a30-68a3-4afd-8175-f9f4b1659cd0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.013942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 400/817] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:34 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:35.014731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e34441-4346-4b01-ad17-efdcd535a245 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-87a07a30-68a3-4afd-8175-f9f4b1659cd0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:35.028287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:35.029711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:35.030830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:35.030830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:35.030830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:35.030830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:35.030830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:35.030830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:35.030830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:35.030830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:35.030830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:35.030830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:35.030830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 418/818] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 12:20:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:35.030830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a07a30-68a3-4afd-8175-f9f4b1659cd0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.122283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.126737 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e34441-4346-4b01-ad17-efdcd535a245 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.134407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.144308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a07a30-68a3-4afd-8175-f9f4b1659cd0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.163625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb635479-1ad4-4b51-ba08-889fdfbc35c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.164463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635479-1ad4-4b51-ba08-889fdfbc35c6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.164902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635479-1ad4-4b51-ba08-889fdfbc35c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.165301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635479-1ad4-4b51-ba08-889fdfbc35c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.176188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb635479-1ad4-4b51-ba08-889fdfbc35c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.178099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb635479-1ad4-4b51-ba08-889fdfbc35c6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.180730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d5222-d7c7-4011-b70a-145b3c317f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.181542 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d5222-d7c7-4011-b70a-145b3c317f52 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.181826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d5222-d7c7-4011-b70a-145b3c317f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.182190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d5222-d7c7-4011-b70a-145b3c317f52 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.182956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d5222-d7c7-4011-b70a-145b3c317f52 None admin] RBAC: Adding query filter params (name=network) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:35.183416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d5222-d7c7-4011-b70a-145b3c317f52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.184922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d5222-d7c7-4011-b70a-145b3c317f52 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.190164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 419/819] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:35.191063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d5222-d7c7-4011-b70a-145b3c317f52 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-eb635479-1ad4-4b51-ba08-889fdfbc35c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:35.195780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:35.196757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:35.197727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:35.197727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:35.197727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:35.200010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 401/820] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:35.201039 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb635479-1ad4-4b51-ba08-889fdfbc35c6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.300510 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d5222-d7c7-4011-b70a-145b3c317f52 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.306793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb635479-1ad4-4b51-ba08-889fdfbc35c6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.338116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac9ce9-2677-4f17-9fd1-ca6f9577e844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.339140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac9ce9-2677-4f17-9fd1-ca6f9577e844 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.339649 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac9ce9-2677-4f17-9fd1-ca6f9577e844 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.340040 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac9ce9-2677-4f17-9fd1-ca6f9577e844 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.340985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac9ce9-2677-4f17-9fd1-ca6f9577e844 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:35.341422 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac9ce9-2677-4f17-9fd1-ca6f9577e844 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.343012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac9ce9-2677-4f17-9fd1-ca6f9577e844 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.346057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e3ed4-7683-4e75-9096-036eaf294b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.346901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e3ed4-7683-4e75-9096-036eaf294b0c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.347374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e3ed4-7683-4e75-9096-036eaf294b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.347844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e3ed4-7683-4e75-9096-036eaf294b0c None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.348678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e3ed4-7683-4e75-9096-036eaf294b0c None admin] RBAC: Adding query filter params (type=network) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:35.349181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e3ed4-7683-4e75-9096-036eaf294b0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.350755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e3ed4-7683-4e75-9096-036eaf294b0c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.356976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 402/821] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:35.358138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e3ed4-7683-4e75-9096-036eaf294b0c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.365365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 420/822] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:35.366262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac9ce9-2677-4f17-9fd1-ca6f9577e844 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.368910 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.385383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.466831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e3ed4-7683-4e75-9096-036eaf294b0c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.476148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac9ce9-2677-4f17-9fd1-ca6f9577e844 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.504040 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950d0b2-15f2-40c5-9f9e-c359616a21c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.505022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950d0b2-15f2-40c5-9f9e-c359616a21c9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.505517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950d0b2-15f2-40c5-9f9e-c359616a21c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.505973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950d0b2-15f2-40c5-9f9e-c359616a21c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.506816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950d0b2-15f2-40c5-9f9e-c359616a21c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:35.507301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950d0b2-15f2-40c5-9f9e-c359616a21c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.509157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950d0b2-15f2-40c5-9f9e-c359616a21c9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.512225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6cea4-9c8c-4dac-b388-a05b90f7a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.513014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6cea4-9c8c-4dac-b388-a05b90f7a58e None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.513268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6cea4-9c8c-4dac-b388-a05b90f7a58e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.513609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6cea4-9c8c-4dac-b388-a05b90f7a58e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.514335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6cea4-9c8c-4dac-b388-a05b90f7a58e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.515772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6cea4-9c8c-4dac-b388-a05b90f7a58e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.524437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 403/823] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:35.525501 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950d0b2-15f2-40c5-9f9e-c359616a21c9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.545679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 421/824] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:35 2022] POST /identity/v3/endpoints => generated 337 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:35.547238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6cea4-9c8c-4dac-b388-a05b90f7a58e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.615584 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.618089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.636171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950d0b2-15f2-40c5-9f9e-c359616a21c9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.674218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6cea4-9c8c-4dac-b388-a05b90f7a58e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.674899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbb467-fce7-444c-8f59-80bf8736584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.675669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbb467-fce7-444c-8f59-80bf8736584a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.676035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbb467-fce7-444c-8f59-80bf8736584a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.676396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbb467-fce7-444c-8f59-80bf8736584a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.687964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbb467-fce7-444c-8f59-80bf8736584a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.689518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbb467-fce7-444c-8f59-80bf8736584a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-04dbb467-fce7-444c-8f59-80bf8736584a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:35.705686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:35.706919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:35.708019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:35.708019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:35.708019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:35.709212 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 404/825] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:35.717851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbb467-fce7-444c-8f59-80bf8736584a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.724058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f366175-b8c7-4dfd-b84b-39aa441f12ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.725011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f366175-b8c7-4dfd-b84b-39aa441f12ec None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.726331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f366175-b8c7-4dfd-b84b-39aa441f12ec None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.726768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f366175-b8c7-4dfd-b84b-39aa441f12ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.742936 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f366175-b8c7-4dfd-b84b-39aa441f12ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.744737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f366175-b8c7-4dfd-b84b-39aa441f12ec None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-8f366175-b8c7-4dfd-b84b-39aa441f12ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:35.768304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:35.769482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:35.770433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:35.770433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:35.770433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:35.770433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:35.770433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:35.770433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:35.770433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:35.770433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:35.773473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 422/826] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:35.785713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f366175-b8c7-4dfd-b84b-39aa441f12ec None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.880981 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbb467-fce7-444c-8f59-80bf8736584a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.913773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f366175-b8c7-4dfd-b84b-39aa441f12ec None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.930086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.935912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafcf5e-426a-4259-90ec-3d1bdf69dedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.936530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafcf5e-426a-4259-90ec-3d1bdf69dedc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.936838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafcf5e-426a-4259-90ec-3d1bdf69dedc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.937108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafcf5e-426a-4259-90ec-3d1bdf69dedc None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.938459 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafcf5e-426a-4259-90ec-3d1bdf69dedc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:35.938763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafcf5e-426a-4259-90ec-3d1bdf69dedc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.940837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafcf5e-426a-4259-90ec-3d1bdf69dedc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.943199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.958269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68425fd-8ff8-499f-85b5-59169391037c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.958884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68425fd-8ff8-499f-85b5-59169391037c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.959160 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68425fd-8ff8-499f-85b5-59169391037c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.959429 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68425fd-8ff8-499f-85b5-59169391037c None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.960076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 405/827] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:35.960192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68425fd-8ff8-499f-85b5-59169391037c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:35.960500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68425fd-8ff8-499f-85b5-59169391037c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.962066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68425fd-8ff8-499f-85b5-59169391037c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.968038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafcf5e-426a-4259-90ec-3d1bdf69dedc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.985024 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 423/828] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:35.985024 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68425fd-8ff8-499f-85b5-59169391037c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.100812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafcf5e-426a-4259-90ec-3d1bdf69dedc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.101508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68425fd-8ff8-499f-85b5-59169391037c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.135354 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b663cce-7862-48ca-8a8f-3ff1c8853dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.135812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d4a3c-39f7-4b6e-8c91-231213d9a6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.136023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b663cce-7862-48ca-8a8f-3ff1c8853dbf None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.136297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b663cce-7862-48ca-8a8f-3ff1c8853dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.136391 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d4a3c-39f7-4b6e-8c91-231213d9a6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.136566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b663cce-7862-48ca-8a8f-3ff1c8853dbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.136693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d4a3c-39f7-4b6e-8c91-231213d9a6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.136877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d4a3c-39f7-4b6e-8c91-231213d9a6f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.137597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d4a3c-39f7-4b6e-8c91-231213d9a6f8 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=e65026e01df242e7be11ed24214a4efa) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:36.137901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d4a3c-39f7-4b6e-8c91-231213d9a6f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:36.139334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d4a3c-39f7-4b6e-8c91-231213d9a6f8 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:36.143493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b663cce-7862-48ca-8a8f-3ff1c8853dbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:36.144964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b663cce-7862-48ca-8a8f-3ff1c8853dbf None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:36.148816 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 424/829] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/role_assignments?user.id=e65026e01df242e7be11ed24214a4efa&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:36.151634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b245357-e024-471b-8c2b-3ee83470c97e None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.152185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b245357-e024-471b-8c2b-3ee83470c97e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.152595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b245357-e024-471b-8c2b-3ee83470c97e None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-5b663cce-7862-48ca-8a8f-3ff1c8853dbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:36.158614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:36.159733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:36.160713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:36.160713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:36.160713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:36.160713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:36.160713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:36.160713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:36.160713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:36.160713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:36.161564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 425/830] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:36 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:36.162751 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 406/831] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:36.163355 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135353b6-73ad-4e3e-bc92-63dfe3ac1759 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.163721 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b663cce-7862-48ca-8a8f-3ff1c8853dbf None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.163978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135353b6-73ad-4e3e-bc92-63dfe3ac1759 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.164486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135353b6-73ad-4e3e-bc92-63dfe3ac1759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.193803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-135353b6-73ad-4e3e-bc92-63dfe3ac1759 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:36.195918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-135353b6-73ad-4e3e-bc92-63dfe3ac1759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:36.233069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:36.241385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:36.292409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 426/832] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:36.293258 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b663cce-7862-48ca-8a8f-3ff1c8853dbf None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.329315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ff2b9-a78d-4874-bcd4-bfcc516b4141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.329924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ff2b9-a78d-4874-bcd4-bfcc516b4141 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.330195 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ff2b9-a78d-4874-bcd4-bfcc516b4141 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.330472 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ff2b9-a78d-4874-bcd4-bfcc516b4141 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.331163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ff2b9-a78d-4874-bcd4-bfcc516b4141 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:36.331460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ff2b9-a78d-4874-bcd4-bfcc516b4141 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:36.333089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ff2b9-a78d-4874-bcd4-bfcc516b4141 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:36.348856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 407/833] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:36.355540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135353b6-73ad-4e3e-bc92-63dfe3ac1759 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.462099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135353b6-73ad-4e3e-bc92-63dfe3ac1759 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.501831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf27c3c-9d2e-4bde-8c26-d992d2af7998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.502898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf27c3c-9d2e-4bde-8c26-d992d2af7998 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.503374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf27c3c-9d2e-4bde-8c26-d992d2af7998 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.503837 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf27c3c-9d2e-4bde-8c26-d992d2af7998 None admin] PATH_INFO: `/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/34a3cadee8ea4b3e9dff0bc23dfe9dc5/roles/248565f1fe7849c5b05a207eb214cbb7` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.518959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf27c3c-9d2e-4bde-8c26-d992d2af7998 None admin] RBAC: Authorizing `identity:create_grant(project_id=70a5f9cf0499405d82f5f5a47c2877eb, user_id=34a3cadee8ea4b3e9dff0bc23dfe9dc5, role_id=248565f1fe7849c5b05a207eb214cbb7)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:36.520753 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf27c3c-9d2e-4bde-8c26-d992d2af7998 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:36.540768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 427/834] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:36 2022] PUT /identity/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/34a3cadee8ea4b3e9dff0bc23dfe9dc5/roles/248565f1fe7849c5b05a207eb214cbb7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:36.562704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ff2b9-a78d-4874-bcd4-bfcc516b4141 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.681866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ff2b9-a78d-4874-bcd4-bfcc516b4141 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.699502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:36.723186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb30d2e-f356-4290-9d45-12906b27a1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.724194 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb30d2e-f356-4290-9d45-12906b27a1de None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.724683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb30d2e-f356-4290-9d45-12906b27a1de None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.725151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb30d2e-f356-4290-9d45-12906b27a1de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.740168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb30d2e-f356-4290-9d45-12906b27a1de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-bdb30d2e-f356-4290-9d45-12906b27a1de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:36.745341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:36.746235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:36.747082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:36.747082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:36.747082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:36.747082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:36.747082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:36.747082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:36.747082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:36.747082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:36.747082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:36.747082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:36.747082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:36.747082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:36.749507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 408/835] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:36.755336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf27c3c-9d2e-4bde-8c26-d992d2af7998 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.779309 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:36.796030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b45c2-69c1-4978-abc4-a7b1d44b52c1 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.796331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b45c2-69c1-4978-abc4-a7b1d44b52c1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.796576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b45c2-69c1-4978-abc4-a7b1d44b52c1 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.798224 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 409/836] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:36.803338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13852f-33fd-439d-b8c3-a11fd8292554 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.803624 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13852f-33fd-439d-b8c3-a11fd8292554 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.803902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13852f-33fd-439d-b8c3-a11fd8292554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.825198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7d13852f-33fd-439d-b8c3-a11fd8292554 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:36.827284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7d13852f-33fd-439d-b8c3-a11fd8292554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:36.865558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf27c3c-9d2e-4bde-8c26-d992d2af7998 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.900284 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa210a-7c09-4524-899d-0b72c31f1afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.900922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa210a-7c09-4524-899d-0b72c31f1afb None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.901229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa210a-7c09-4524-899d-0b72c31f1afb None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.901710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa210a-7c09-4524-899d-0b72c31f1afb None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.902505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa210a-7c09-4524-899d-0b72c31f1afb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:36.902808 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa210a-7c09-4524-899d-0b72c31f1afb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:36.904195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa210a-7c09-4524-899d-0b72c31f1afb None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:36.910768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 410/837] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:36.920675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 428/838] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:36.926283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13852f-33fd-439d-b8c3-a11fd8292554 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.030407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13852f-33fd-439d-b8c3-a11fd8292554 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.033757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.070984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288356ee-7b13-4cc0-9bea-afbea0ae56ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.071791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288356ee-7b13-4cc0-9bea-afbea0ae56ae None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.072169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288356ee-7b13-4cc0-9bea-afbea0ae56ae None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.072529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288356ee-7b13-4cc0-9bea-afbea0ae56ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.084700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288356ee-7b13-4cc0-9bea-afbea0ae56ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.086351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288356ee-7b13-4cc0-9bea-afbea0ae56ae None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-288356ee-7b13-4cc0-9bea-afbea0ae56ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:37.102274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:37.103484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:37.104734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:37.104734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:37.104734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:37.106138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 411/839] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:37.114889 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa210a-7c09-4524-899d-0b72c31f1afb None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.200241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.219591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288356ee-7b13-4cc0-9bea-afbea0ae56ae None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.220025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa210a-7c09-4524-899d-0b72c31f1afb None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.254758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb20107-cc0f-4c1a-9c6b-7e2d6ff336c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.255278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb20107-cc0f-4c1a-9c6b-7e2d6ff336c4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.255551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb20107-cc0f-4c1a-9c6b-7e2d6ff336c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.255816 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb20107-cc0f-4c1a-9c6b-7e2d6ff336c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.256463 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb20107-cc0f-4c1a-9c6b-7e2d6ff336c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:37.256799 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb20107-cc0f-4c1a-9c6b-7e2d6ff336c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.258170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb20107-cc0f-4c1a-9c6b-7e2d6ff336c4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.270867 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 429/840] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:37.276550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb20107-cc0f-4c1a-9c6b-7e2d6ff336c4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.332048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288356ee-7b13-4cc0-9bea-afbea0ae56ae None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.370604 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1dc73-c50e-418b-8701-1996d2f8291c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.371244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1dc73-c50e-418b-8701-1996d2f8291c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.371519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1dc73-c50e-418b-8701-1996d2f8291c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.371793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1dc73-c50e-418b-8701-1996d2f8291c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.372485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1dc73-c50e-418b-8701-1996d2f8291c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.374156 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1dc73-c50e-418b-8701-1996d2f8291c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.378911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb20107-cc0f-4c1a-9c6b-7e2d6ff336c4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-36a1dc73-c50e-418b-8701-1996d2f8291c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:37.383552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:37.385884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:37.387871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:37.389730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 412/841] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:37.396694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1dc73-c50e-418b-8701-1996d2f8291c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.415865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d25ec-faa3-468d-a01a-6a02b0106b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.416501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d25ec-faa3-468d-a01a-6a02b0106b00 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.416805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d25ec-faa3-468d-a01a-6a02b0106b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.417362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d25ec-faa3-468d-a01a-6a02b0106b00 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.417571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.425324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d25ec-faa3-468d-a01a-6a02b0106b00 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.426854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d25ec-faa3-468d-a01a-6a02b0106b00 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.429598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-483d25ec-faa3-468d-a01a-6a02b0106b00 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:37.440276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:37.441470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:37.442651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:37.442651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:37.442651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:37.442651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:37.442651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:37.442651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:37.442651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:37.442651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:37.442651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:37.442651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:37.450102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 430/842] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:37.451090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d25ec-faa3-468d-a01a-6a02b0106b00 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.511583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1dc73-c50e-418b-8701-1996d2f8291c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.548325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bd493-0f92-4a03-8fb6-1b9c4eb72de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.549358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bd493-0f92-4a03-8fb6-1b9c4eb72de1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.549901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bd493-0f92-4a03-8fb6-1b9c4eb72de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.550363 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bd493-0f92-4a03-8fb6-1b9c4eb72de1 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.551166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644bd493-0f92-4a03-8fb6-1b9c4eb72de1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:37.551657 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644bd493-0f92-4a03-8fb6-1b9c4eb72de1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.552632 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d25ec-faa3-468d-a01a-6a02b0106b00 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.553435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644bd493-0f92-4a03-8fb6-1b9c4eb72de1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.563423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 413/843] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:37.565541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bd493-0f92-4a03-8fb6-1b9c4eb72de1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.588106 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19bda-5a3b-48b3-b967-a0a5a99fdf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.588908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19bda-5a3b-48b3-b967-a0a5a99fdf0f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.589279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19bda-5a3b-48b3-b967-a0a5a99fdf0f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.589549 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19bda-5a3b-48b3-b967-a0a5a99fdf0f None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.590279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e19bda-5a3b-48b3-b967-a0a5a99fdf0f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:37.590576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e19bda-5a3b-48b3-b967-a0a5a99fdf0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.592184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e19bda-5a3b-48b3-b967-a0a5a99fdf0f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.614996 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 431/844] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:37.616057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19bda-5a3b-48b3-b967-a0a5a99fdf0f None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.661572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.676478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.682145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bd493-0f92-4a03-8fb6-1b9c4eb72de1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.722470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19bda-5a3b-48b3-b967-a0a5a99fdf0f None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.726706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ef709-cb83-46bb-9c93-a24bb9cd1acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.727535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ef709-cb83-46bb-9c93-a24bb9cd1acc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.727983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ef709-cb83-46bb-9c93-a24bb9cd1acc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.728450 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ef709-cb83-46bb-9c93-a24bb9cd1acc None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.729289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ef709-cb83-46bb-9c93-a24bb9cd1acc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:37.729814 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ef709-cb83-46bb-9c93-a24bb9cd1acc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.731471 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ef709-cb83-46bb-9c93-a24bb9cd1acc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.744835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 414/845] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:37.760239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b125967-a699-48fd-86e4-2c8ab7e85d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.765642 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b125967-a699-48fd-86e4-2c8ab7e85d1a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.765642 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b125967-a699-48fd-86e4-2c8ab7e85d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.765642 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b125967-a699-48fd-86e4-2c8ab7e85d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.843529 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b125967-a699-48fd-86e4-2c8ab7e85d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.845088 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b125967-a699-48fd-86e4-2c8ab7e85d1a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.912606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.958820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 432/846] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 12:20:37.965638 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ef709-cb83-46bb-9c93-a24bb9cd1acc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:38.074964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ef709-cb83-46bb-9c93-a24bb9cd1acc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:38.109705 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023e909-0af4-4a04-8305-8f87866b64b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:38.110334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023e909-0af4-4a04-8305-8f87866b64b1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.110605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023e909-0af4-4a04-8305-8f87866b64b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.110870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023e909-0af4-4a04-8305-8f87866b64b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.122389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023e909-0af4-4a04-8305-8f87866b64b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:38.123866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023e909-0af4-4a04-8305-8f87866b64b1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:38.132601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f023e909-0af4-4a04-8305-8f87866b64b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:38.140250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:38.142371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:38.144486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:38.144486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:38.144486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:38.144486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 415/847] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:38.147605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b125967-a699-48fd-86e4-2c8ab7e85d1a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:38.262586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b125967-a699-48fd-86e4-2c8ab7e85d1a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:38.278706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e55a7-bad7-4bbd-a9e6-dab4ae3c8016 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.279102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e55a7-bad7-4bbd-a9e6-dab4ae3c8016 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.279477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e55a7-bad7-4bbd-a9e6-dab4ae3c8016 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.281191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 416/848] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:38.287577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a67db2-c51f-487e-8e79-bec846f77d33 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.287577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a67db2-c51f-487e-8e79-bec846f77d33 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.287577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a67db2-c51f-487e-8e79-bec846f77d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.305916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4056bda-2e23-47b9-ab95-a688fc531378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:38.306858 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4056bda-2e23-47b9-ab95-a688fc531378 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.307329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4056bda-2e23-47b9-ab95-a688fc531378 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.307820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4056bda-2e23-47b9-ab95-a688fc531378 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.308745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4056bda-2e23-47b9-ab95-a688fc531378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:38.309208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4056bda-2e23-47b9-ab95-a688fc531378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:38.309448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c0a67db2-c51f-487e-8e79-bec846f77d33 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:38.310636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c0a67db2-c51f-487e-8e79-bec846f77d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:38.311112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4056bda-2e23-47b9-ab95-a688fc531378 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:38.327311 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 433/849] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:38.333445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4056bda-2e23-47b9-ab95-a688fc531378 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:38.359694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:38.402416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 417/850] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:38.465506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4056bda-2e23-47b9-ab95-a688fc531378 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:38.502515 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46436bec-645e-48dc-91f9-513d4142d838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:38.503430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46436bec-645e-48dc-91f9-513d4142d838 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.503877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46436bec-645e-48dc-91f9-513d4142d838 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.504297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46436bec-645e-48dc-91f9-513d4142d838 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.511492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46436bec-645e-48dc-91f9-513d4142d838 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:38.513197 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46436bec-645e-48dc-91f9-513d4142d838 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-46436bec-645e-48dc-91f9-513d4142d838 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:38.524905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:38.525997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:38.527367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:38.527367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:38.527367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:38.527367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:38.527367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:38.527367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:38.527367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:38.527367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:38.529493 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 434/851] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:38.537367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a67db2-c51f-487e-8e79-bec846f77d33 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:38.562196 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a76f8-d223-4333-8b9d-6e8bcc6a630b None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.562495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a76f8-d223-4333-8b9d-6e8bcc6a630b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.562756 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a76f8-d223-4333-8b9d-6e8bcc6a630b None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.564338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 435/852] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:38.569124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7541c69-9d07-4b77-84a1-a7b05a3c005f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.569401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7541c69-9d07-4b77-84a1-a7b05a3c005f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.569673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7541c69-9d07-4b77-84a1-a7b05a3c005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.588569 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b7541c69-9d07-4b77-84a1-a7b05a3c005f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:38.589949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b7541c69-9d07-4b77-84a1-a7b05a3c005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:38.641838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a67db2-c51f-487e-8e79-bec846f77d33 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:38.672934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 436/853] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:38.673852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7541c69-9d07-4b77-84a1-a7b05a3c005f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:38.679939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:38.686338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:38.687883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9cf64-3304-42bd-a4c8-9c422b2d9fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:38.688814 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9cf64-3304-42bd-a4c8-9c422b2d9fff None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.689333 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9cf64-3304-42bd-a4c8-9c422b2d9fff None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.689668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9cf64-3304-42bd-a4c8-9c422b2d9fff None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.690506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9cf64-3304-42bd-a4c8-9c422b2d9fff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:38.690921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9cf64-3304-42bd-a4c8-9c422b2d9fff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:38.693000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9cf64-3304-42bd-a4c8-9c422b2d9fff None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:38.708783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 418/854] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:38.718124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9cf64-3304-42bd-a4c8-9c422b2d9fff None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:38.794070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7541c69-9d07-4b77-84a1-a7b05a3c005f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:38.832447 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f950f-4bcc-4adf-bcf4-a8b7bfa3fe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:38.833379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f950f-4bcc-4adf-bcf4-a8b7bfa3fe22 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.833798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f950f-4bcc-4adf-bcf4-a8b7bfa3fe22 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.834260 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f950f-4bcc-4adf-bcf4-a8b7bfa3fe22 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.835109 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f950f-4bcc-4adf-bcf4-a8b7bfa3fe22 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:38.836714 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f950f-4bcc-4adf-bcf4-a8b7bfa3fe22 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-163f950f-4bcc-4adf-bcf4-a8b7bfa3fe22 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:38.847251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:38.848393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:38.849519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:38.850231 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9cf64-3304-42bd-a4c8-9c422b2d9fff None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:38.852315 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 437/855] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Aug 24 12:20:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:38.857977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f950f-4bcc-4adf-bcf4-a8b7bfa3fe22 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:38.891852 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbb61e-4ee7-4e46-a051-38a0c354e161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:38.892802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbb61e-4ee7-4e46-a051-38a0c354e161 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.893253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbb61e-4ee7-4e46-a051-38a0c354e161 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.893777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbb61e-4ee7-4e46-a051-38a0c354e161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.894698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbb61e-4ee7-4e46-a051-38a0c354e161 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=bfec040b09c9415b9e89cd28c67aa5df) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:38.895049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbb61e-4ee7-4e46-a051-38a0c354e161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:38.896546 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbb61e-4ee7-4e46-a051-38a0c354e161 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:38.903192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 419/856] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:38 2022] GET /identity/v3/role_assignments?user.id=bfec040b09c9415b9e89cd28c67aa5df&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:38.941143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:38.959280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbb61e-4ee7-4e46-a051-38a0c354e161 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:38.989260 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f950f-4bcc-4adf-bcf4-a8b7bfa3fe22 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:39.025014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:39.027018 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55153-c434-4a07-b04e-08aec83eacba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:39.028100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55153-c434-4a07-b04e-08aec83eacba None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.028558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55153-c434-4a07-b04e-08aec83eacba None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.029038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55153-c434-4a07-b04e-08aec83eacba None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.029830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe55153-c434-4a07-b04e-08aec83eacba None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:39.030297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe55153-c434-4a07-b04e-08aec83eacba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:39.031899 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe55153-c434-4a07-b04e-08aec83eacba None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:39.042822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 438/857] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:39.044081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55153-c434-4a07-b04e-08aec83eacba None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:39.084264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbb61e-4ee7-4e46-a051-38a0c354e161 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:39.130018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be6570-7ce4-457d-8ff1-eb9191b96cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:39.130593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be6570-7ce4-457d-8ff1-eb9191b96cfe None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.130865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be6570-7ce4-457d-8ff1-eb9191b96cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.131129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be6570-7ce4-457d-8ff1-eb9191b96cfe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.144587 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be6570-7ce4-457d-8ff1-eb9191b96cfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b2be6570-7ce4-457d-8ff1-eb9191b96cfe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:39.150012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:39.150951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:39.151982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 420/858] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:39.157778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be6570-7ce4-457d-8ff1-eb9191b96cfe None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:39.179079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55153-c434-4a07-b04e-08aec83eacba None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:39.222965 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20464f-f392-43a0-bd85-614a01130efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:39.223693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20464f-f392-43a0-bd85-614a01130efd None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.224047 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20464f-f392-43a0-bd85-614a01130efd None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.224386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20464f-f392-43a0-bd85-614a01130efd None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.225599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20464f-f392-43a0-bd85-614a01130efd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:39.225983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20464f-f392-43a0-bd85-614a01130efd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:39.227737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20464f-f392-43a0-bd85-614a01130efd None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:39.231019 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:39.240809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 439/859] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:39.286002 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be6570-7ce4-457d-8ff1-eb9191b96cfe None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:39.302170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:39.325931 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6957e-398a-468f-b94e-1f01f84b2c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:39.326905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6957e-398a-468f-b94e-1f01f84b2c78 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.327355 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6957e-398a-468f-b94e-1f01f84b2c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.327795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6957e-398a-468f-b94e-1f01f84b2c78 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.328740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6957e-398a-468f-b94e-1f01f84b2c78 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:39.329219 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6957e-398a-468f-b94e-1f01f84b2c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:39.330911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6957e-398a-468f-b94e-1f01f84b2c78 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:39.346544 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 421/860] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:39.348541 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20464f-f392-43a0-bd85-614a01130efd None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:39.463061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20464f-f392-43a0-bd85-614a01130efd None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:39.500229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d115fc3-59e5-47ed-9d73-824d1012a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:39.501659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d115fc3-59e5-47ed-9d73-824d1012a274 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.502111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d115fc3-59e5-47ed-9d73-824d1012a274 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.502561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d115fc3-59e5-47ed-9d73-824d1012a274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.515457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d115fc3-59e5-47ed-9d73-824d1012a274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:39.517269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d115fc3-59e5-47ed-9d73-824d1012a274 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6d115fc3-59e5-47ed-9d73-824d1012a274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:39.532747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:39.533742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:39.534667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:39.534667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:39.534667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:39.540787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 440/861] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:39.542666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6957e-398a-468f-b94e-1f01f84b2c78 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:39.648248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6957e-398a-468f-b94e-1f01f84b2c78 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:39.683312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4edee-c357-48a2-8926-97b158a89e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:39.683920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4edee-c357-48a2-8926-97b158a89e1b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.684199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4edee-c357-48a2-8926-97b158a89e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.684471 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4edee-c357-48a2-8926-97b158a89e1b None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.685151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4edee-c357-48a2-8926-97b158a89e1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:39.685452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4edee-c357-48a2-8926-97b158a89e1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:39.686843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4edee-c357-48a2-8926-97b158a89e1b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:39.699678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 422/862] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:39.705300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d115fc3-59e5-47ed-9d73-824d1012a274 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:39.763021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:39.818921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d115fc3-59e5-47ed-9d73-824d1012a274 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:39.835518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da613051-fdb8-4122-9391-2e09b61e4449 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.835830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da613051-fdb8-4122-9391-2e09b61e4449 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.836069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da613051-fdb8-4122-9391-2e09b61e4449 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.838239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 423/863] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:39.844323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275e413-3a38-47e5-a219-ea11184aa88c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.844785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275e413-3a38-47e5-a219-ea11184aa88c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.845165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275e413-3a38-47e5-a219-ea11184aa88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.860051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d607a-944e-4f33-b793-79ba8bb9aad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:39.860977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d607a-944e-4f33-b793-79ba8bb9aad9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.861414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d607a-944e-4f33-b793-79ba8bb9aad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.861824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d607a-944e-4f33-b793-79ba8bb9aad9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.865043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3275e413-3a38-47e5-a219-ea11184aa88c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:39.865993 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3275e413-3a38-47e5-a219-ea11184aa88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:39.870648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d607a-944e-4f33-b793-79ba8bb9aad9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:39.872272 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d607a-944e-4f33-b793-79ba8bb9aad9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-454d607a-944e-4f33-b793-79ba8bb9aad9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:39.886818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:39.888012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:39.889184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:39.889184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:39.889184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:39.889184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:39.889184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:39.889184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:39.889184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:39.889184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:39.889184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:39.889184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:39.891969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 441/864] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 12:20:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:39.900671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d607a-944e-4f33-b793-79ba8bb9aad9 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:39.900857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:39.947090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 424/865] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:40.011574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d607a-944e-4f33-b793-79ba8bb9aad9 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.025022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:40.046627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fef9ea-6ac1-47ab-a50f-ab1d085237df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.047233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fef9ea-6ac1-47ab-a50f-ab1d085237df None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.047505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fef9ea-6ac1-47ab-a50f-ab1d085237df None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.047768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fef9ea-6ac1-47ab-a50f-ab1d085237df None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.048493 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fef9ea-6ac1-47ab-a50f-ab1d085237df None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:40.048894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fef9ea-6ac1-47ab-a50f-ab1d085237df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.050341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fef9ea-6ac1-47ab-a50f-ab1d085237df None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.071138 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 442/866] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 24 12:20:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:40.077032 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275e413-3a38-47e5-a219-ea11184aa88c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.179609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275e413-3a38-47e5-a219-ea11184aa88c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.215114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1db215-ac8e-4723-8555-061a237cfe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.215716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1db215-ac8e-4723-8555-061a237cfe93 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.215985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1db215-ac8e-4723-8555-061a237cfe93 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.216249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1db215-ac8e-4723-8555-061a237cfe93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.229689 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fef9ea-6ac1-47ab-a50f-ab1d085237df None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.258126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:40.307201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1db215-ac8e-4723-8555-061a237cfe93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.308887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1db215-ac8e-4723-8555-061a237cfe93 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.345005 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fef9ea-6ac1-47ab-a50f-ab1d085237df None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.385727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9a875-9147-404a-9192-54b5cb9b4781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.386216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9a875-9147-404a-9192-54b5cb9b4781 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.386509 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9a875-9147-404a-9192-54b5cb9b4781 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.386780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9a875-9147-404a-9192-54b5cb9b4781 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.387435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9a875-9147-404a-9192-54b5cb9b4781 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.388857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9a875-9147-404a-9192-54b5cb9b4781 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.409189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 443/867] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:40 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:40.438911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 425/868] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 12:20:40.444602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9a875-9147-404a-9192-54b5cb9b4781 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.484947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:40.575097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9a875-9147-404a-9192-54b5cb9b4781 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.611635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febc611-43d3-4dd6-ad2e-2a38731ea5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.612266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febc611-43d3-4dd6-ad2e-2a38731ea5de None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.612539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febc611-43d3-4dd6-ad2e-2a38731ea5de None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.612834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febc611-43d3-4dd6-ad2e-2a38731ea5de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.623950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febc611-43d3-4dd6-ad2e-2a38731ea5de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.625492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febc611-43d3-4dd6-ad2e-2a38731ea5de None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-2febc611-43d3-4dd6-ad2e-2a38731ea5de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:40.641370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:40.642344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:40.643242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:40.643242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:40.643242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:40.643242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 444/869] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:40.649921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1db215-ac8e-4723-8555-061a237cfe93 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.719319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:40.759845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1db215-ac8e-4723-8555-061a237cfe93 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.795827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362c2bf-9f2c-4694-8b89-1238401273b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.796440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362c2bf-9f2c-4694-8b89-1238401273b8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.796740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362c2bf-9f2c-4694-8b89-1238401273b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.797007 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362c2bf-9f2c-4694-8b89-1238401273b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.797659 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362c2bf-9f2c-4694-8b89-1238401273b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:40.797957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362c2bf-9f2c-4694-8b89-1238401273b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.799343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362c2bf-9f2c-4694-8b89-1238401273b8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.812135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 426/870] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:40.817686 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febc611-43d3-4dd6-ad2e-2a38731ea5de None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.900631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:40.928092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febc611-43d3-4dd6-ad2e-2a38731ea5de None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.964249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127899a-6ec1-4435-80df-78d523ddd446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.964903 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127899a-6ec1-4435-80df-78d523ddd446 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.965181 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127899a-6ec1-4435-80df-78d523ddd446 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.965453 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127899a-6ec1-4435-80df-78d523ddd446 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.969796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015862-5724-45e3-951f-3c5b352c8934 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.970078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015862-5724-45e3-951f-3c5b352c8934 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.970347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015862-5724-45e3-951f-3c5b352c8934 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.971956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 427/871] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:40.981293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127899a-6ec1-4435-80df-78d523ddd446 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.981293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127899a-6ec1-4435-80df-78d523ddd446 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.981770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4b6c2-479f-438e-95c2-2bc80ad74309 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.981770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4b6c2-479f-438e-95c2-2bc80ad74309 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.982063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4b6c2-479f-438e-95c2-2bc80ad74309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-5127899a-6ec1-4435-80df-78d523ddd446 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:41.003113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:41.005230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:41.007007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:41.007007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:41.007007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:41.007007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:41.007007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:41.007007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:41.007007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:41.007007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:41.007541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-04a4b6c2-479f-438e-95c2-2bc80ad74309 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:41.007541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-04a4b6c2-479f-438e-95c2-2bc80ad74309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:41.007696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 445/872] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:41.012003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127899a-6ec1-4435-80df-78d523ddd446 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.080771 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 428/873] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:41.133024 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127899a-6ec1-4435-80df-78d523ddd446 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.168133 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e43d6-1fd9-4c9b-a8d1-9d9065da65a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.168987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e43d6-1fd9-4c9b-a8d1-9d9065da65a4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.169411 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e43d6-1fd9-4c9b-a8d1-9d9065da65a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.169845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e43d6-1fd9-4c9b-a8d1-9d9065da65a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.170700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e43d6-1fd9-4c9b-a8d1-9d9065da65a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:41.171147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e43d6-1fd9-4c9b-a8d1-9d9065da65a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.172747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e43d6-1fd9-4c9b-a8d1-9d9065da65a4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.180423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:41.185949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 446/874] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:41.192684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4b6c2-479f-438e-95c2-2bc80ad74309 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.251645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:41.297176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4b6c2-479f-438e-95c2-2bc80ad74309 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.332868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4253d34-a056-46f9-9a5e-afcd61b04032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.333477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4253d34-a056-46f9-9a5e-afcd61b04032 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.333806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4253d34-a056-46f9-9a5e-afcd61b04032 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.334076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4253d34-a056-46f9-9a5e-afcd61b04032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.334815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4253d34-a056-46f9-9a5e-afcd61b04032 None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=34a3cadee8ea4b3e9dff0bc23dfe9dc5) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:41.335121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4253d34-a056-46f9-9a5e-afcd61b04032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.336571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4253d34-a056-46f9-9a5e-afcd61b04032 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.345918 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 429/875] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/role_assignments?user.id=34a3cadee8ea4b3e9dff0bc23dfe9dc5&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:41.350644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eb70e-2d91-40e6-8918-8743dae93de7 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.351244 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eb70e-2d91-40e6-8918-8743dae93de7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.351719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eb70e-2d91-40e6-8918-8743dae93de7 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.353699 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 447/876] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:41.357701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4253d34-a056-46f9-9a5e-afcd61b04032 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.363819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c98bb2-ce16-43e4-b20a-52730461be17 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.364115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c98bb2-ce16-43e4-b20a-52730461be17 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.364374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c98bb2-ce16-43e4-b20a-52730461be17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.393821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f2c98bb2-ce16-43e4-b20a-52730461be17 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:41.395644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f2c98bb2-ce16-43e4-b20a-52730461be17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:41.480772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 448/877] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:41 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:41.484326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4253d34-a056-46f9-9a5e-afcd61b04032 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.507441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:41.523496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c54d26-206d-45ea-a3cc-bd975cba5d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.524145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54d26-206d-45ea-a3cc-bd975cba5d4a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.524428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54d26-206d-45ea-a3cc-bd975cba5d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.524732 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54d26-206d-45ea-a3cc-bd975cba5d4a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.525330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c54d26-206d-45ea-a3cc-bd975cba5d4a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.527003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c54d26-206d-45ea-a3cc-bd975cba5d4a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c0c54d26-206d-45ea-a3cc-bd975cba5d4a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:41.536631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:41.537896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:41.538840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 430/878] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:41.546045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c98bb2-ce16-43e4-b20a-52730461be17 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.656177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c98bb2-ce16-43e4-b20a-52730461be17 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.684471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:41.694002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb4559-1d68-479c-8a8b-3d7eab7faa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.694884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb4559-1d68-479c-8a8b-3d7eab7faa03 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.695361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb4559-1d68-479c-8a8b-3d7eab7faa03 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.695898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb4559-1d68-479c-8a8b-3d7eab7faa03 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.696767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb4559-1d68-479c-8a8b-3d7eab7faa03 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:41.697263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb4559-1d68-479c-8a8b-3d7eab7faa03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.699056 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb4559-1d68-479c-8a8b-3d7eab7faa03 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.704310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 449/879] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:41.710275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c54d26-206d-45ea-a3cc-bd975cba5d4a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.762509 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb4559-1d68-479c-8a8b-3d7eab7faa03 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.820401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c54d26-206d-45ea-a3cc-bd975cba5d4a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.864676 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f8629-7f93-484d-8b9f-79eb687f521d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.866084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f8629-7f93-484d-8b9f-79eb687f521d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.866610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f8629-7f93-484d-8b9f-79eb687f521d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.867063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f8629-7f93-484d-8b9f-79eb687f521d None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.868087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f8629-7f93-484d-8b9f-79eb687f521d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:41.868918 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f8629-7f93-484d-8b9f-79eb687f521d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.871121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f8629-7f93-484d-8b9f-79eb687f521d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.875445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb4559-1d68-479c-8a8b-3d7eab7faa03 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.884694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 431/880] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:41.920910 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbd012-887b-4f28-872a-bd9028be3bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.921541 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbd012-887b-4f28-872a-bd9028be3bc1 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.921821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbd012-887b-4f28-872a-bd9028be3bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.922090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbd012-887b-4f28-872a-bd9028be3bc1 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.922769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbd012-887b-4f28-872a-bd9028be3bc1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.924185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbd012-887b-4f28-872a-bd9028be3bc1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.944317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 450/881] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:41 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:42.383306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df98ab-85b6-4a80-9af2-ec2ff4d08b1e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:42.383585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df98ab-85b6-4a80-9af2-ec2ff4d08b1e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:42.384109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df98ab-85b6-4a80-9af2-ec2ff4d08b1e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:42.386010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 432/882] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:42.391663 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade42f1-2052-4a3d-af60-4f2c87eb98a0 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:42.392187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade42f1-2052-4a3d-af60-4f2c87eb98a0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:42.392707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade42f1-2052-4a3d-af60-4f2c87eb98a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:42.411899 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6ade42f1-2052-4a3d-af60-4f2c87eb98a0 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:42.413448 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6ade42f1-2052-4a3d-af60-4f2c87eb98a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:42.463779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:42.493441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 451/883] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:42.757649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df98ab-85b6-4a80-9af2-ec2ff4d08b1e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:42.811741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:42.866515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df98ab-85b6-4a80-9af2-ec2ff4d08b1e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:42.906066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6f073-9da1-4278-ae5e-1b720a903f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:42.906958 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6f073-9da1-4278-ae5e-1b720a903f9c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:42.907375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6f073-9da1-4278-ae5e-1b720a903f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:42.907816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6f073-9da1-4278-ae5e-1b720a903f9c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:42.908543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6f073-9da1-4278-ae5e-1b720a903f9c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:42.910148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6f073-9da1-4278-ae5e-1b720a903f9c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-d8a6f073-9da1-4278-ae5e-1b720a903f9c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:42.919414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:42.920621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:42.921668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:42.924081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 433/884] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:42.929765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade42f1-2052-4a3d-af60-4f2c87eb98a0 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:43.053352 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade42f1-2052-4a3d-af60-4f2c87eb98a0 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:43.092253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970ad94-60a8-4f82-9147-ebf08bffb871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:43.092895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970ad94-60a8-4f82-9147-ebf08bffb871 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.093173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970ad94-60a8-4f82-9147-ebf08bffb871 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.093434 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970ad94-60a8-4f82-9147-ebf08bffb871 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.094269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970ad94-60a8-4f82-9147-ebf08bffb871 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:43.094570 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970ad94-60a8-4f82-9147-ebf08bffb871 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:43.095944 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970ad94-60a8-4f82-9147-ebf08bffb871 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:43.101244 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 452/885] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:43.107116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6f073-9da1-4278-ae5e-1b720a903f9c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:43.211181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6f073-9da1-4278-ae5e-1b720a903f9c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:43.225214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:43.246734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa549f0-ad7f-4ea1-9675-5a4c225cb6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:43.247335 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa549f0-ad7f-4ea1-9675-5a4c225cb6ee None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.247611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa549f0-ad7f-4ea1-9675-5a4c225cb6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.247876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa549f0-ad7f-4ea1-9675-5a4c225cb6ee None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.248508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa549f0-ad7f-4ea1-9675-5a4c225cb6ee None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:43.248835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa549f0-ad7f-4ea1-9675-5a4c225cb6ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:43.250310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa549f0-ad7f-4ea1-9675-5a4c225cb6ee None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:43.255698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 434/886] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:43.261883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970ad94-60a8-4f82-9147-ebf08bffb871 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:43.307858 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:43.368041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970ad94-60a8-4f82-9147-ebf08bffb871 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:43.403563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591e895-b5fd-4e23-a02d-b4017b30104d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:43.404558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591e895-b5fd-4e23-a02d-b4017b30104d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.405061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591e895-b5fd-4e23-a02d-b4017b30104d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.405528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591e895-b5fd-4e23-a02d-b4017b30104d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.406464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591e895-b5fd-4e23-a02d-b4017b30104d None admin] RBAC: Adding query filter params (interface=public, service_id=f90600422d2f4fac8f400e3f80e170dc, region_id=RegionOne) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:43.407037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591e895-b5fd-4e23-a02d-b4017b30104d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:43.408689 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591e895-b5fd-4e23-a02d-b4017b30104d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:43.414724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 453/887] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:43 2022] GET /identity/v3/endpoints?service_id=f90600422d2f4fac8f400e3f80e170dc&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:43.420770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa549f0-ad7f-4ea1-9675-5a4c225cb6ee None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:43.505521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8d0ad-fe77-4e4a-93a4-6b3ca1a81dc5 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.506169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8d0ad-fe77-4e4a-93a4-6b3ca1a81dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.506668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8d0ad-fe77-4e4a-93a4-6b3ca1a81dc5 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.508467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 454/888] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:43.514069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f662f-948e-4563-8711-d6b45e256b70 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.514710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f662f-948e-4563-8711-d6b45e256b70 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.515179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f662f-948e-4563-8711-d6b45e256b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.530058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa549f0-ad7f-4ea1-9675-5a4c225cb6ee None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:43.536728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-102f662f-948e-4563-8711-d6b45e256b70 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:43.538676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-102f662f-948e-4563-8711-d6b45e256b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:43.568848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72debcb1-42dc-4bb9-9c4f-5807e9378b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:43.569402 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72debcb1-42dc-4bb9-9c4f-5807e9378b03 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.569678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72debcb1-42dc-4bb9-9c4f-5807e9378b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.569946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72debcb1-42dc-4bb9-9c4f-5807e9378b03 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.570559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72debcb1-42dc-4bb9-9c4f-5807e9378b03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:43.572062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72debcb1-42dc-4bb9-9c4f-5807e9378b03 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:43.579249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 435/889] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:43 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:20:43.626308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 455/890] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:43.865233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf5e4a-d147-424f-917a-4c071a7f2f2a None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.866242 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf5e4a-d147-424f-917a-4c071a7f2f2a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.866738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf5e4a-d147-424f-917a-4c071a7f2f2a None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.868547 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 436/891] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:43.874297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d66b2d-7eb8-4461-846e-46956d02b38d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.874845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d66b2d-7eb8-4461-846e-46956d02b38d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.875311 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d66b2d-7eb8-4461-846e-46956d02b38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.893055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf5e4a-d147-424f-917a-4c071a7f2f2a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:43.893949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-58d66b2d-7eb8-4461-846e-46956d02b38d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:43.895125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-58d66b2d-7eb8-4461-846e-46956d02b38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:43.927639 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:43.941989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:43.977660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 456/892] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:43.983008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bebda7-6b4c-4473-8540-86c506c1175e None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.983694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bebda7-6b4c-4473-8540-86c506c1175e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.984115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bebda7-6b4c-4473-8540-86c506c1175e None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.991072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 457/893] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:43.992449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65ddd3-cf25-40ce-9d77-f14d7586111d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.993060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65ddd3-cf25-40ce-9d77-f14d7586111d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.993342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65ddd3-cf25-40ce-9d77-f14d7586111d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.016330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0a65ddd3-cf25-40ce-9d77-f14d7586111d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:44.018585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0a65ddd3-cf25-40ce-9d77-f14d7586111d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:44.021193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf5e4a-d147-424f-917a-4c071a7f2f2a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.057694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fa50c-fce4-4cc2-ae72-7f6d8ae7379a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.058712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fa50c-fce4-4cc2-ae72-7f6d8ae7379a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.059137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fa50c-fce4-4cc2-ae72-7f6d8ae7379a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.059637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fa50c-fce4-4cc2-ae72-7f6d8ae7379a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.070979 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fa50c-fce4-4cc2-ae72-7f6d8ae7379a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-315fa50c-fce4-4cc2-ae72-7f6d8ae7379a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:44.076455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:44.077454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:44.078289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:44.078289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:44.078289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:44.078289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:44.078289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:44.078289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:44.078289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:44.078289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:44.078289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:44.078289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:44.078289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:44.078289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:44.080968 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 437/894] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 24 12:20:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:44.086498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fa50c-fce4-4cc2-ae72-7f6d8ae7379a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.108202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 458/895] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:44.203698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:44.207604 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fa50c-fce4-4cc2-ae72-7f6d8ae7379a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.240806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65ddd3-cf25-40ce-9d77-f14d7586111d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.247430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd12f2a-bca6-461d-b2f6-c14b60afdfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.248052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd12f2a-bca6-461d-b2f6-c14b60afdfea None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.248330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd12f2a-bca6-461d-b2f6-c14b60afdfea None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.248599 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd12f2a-bca6-461d-b2f6-c14b60afdfea None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.249307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd12f2a-bca6-461d-b2f6-c14b60afdfea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:44.249610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd12f2a-bca6-461d-b2f6-c14b60afdfea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.251003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd12f2a-bca6-461d-b2f6-c14b60afdfea None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.261261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 438/896] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:44.265870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd12f2a-bca6-461d-b2f6-c14b60afdfea None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.369913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:44.378013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65ddd3-cf25-40ce-9d77-f14d7586111d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.387917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd12f2a-bca6-461d-b2f6-c14b60afdfea None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.413722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f39541-eec9-44e9-8ecd-660759e41501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.414728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f39541-eec9-44e9-8ecd-660759e41501 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.415318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f39541-eec9-44e9-8ecd-660759e41501 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.415667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f39541-eec9-44e9-8ecd-660759e41501 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.416455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f39541-eec9-44e9-8ecd-660759e41501 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.418223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f39541-eec9-44e9-8ecd-660759e41501 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.423785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e939c83-8669-4f27-a7d6-d901e56b2130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.424567 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e939c83-8669-4f27-a7d6-d901e56b2130 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.424998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e939c83-8669-4f27-a7d6-d901e56b2130 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.425338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e939c83-8669-4f27-a7d6-d901e56b2130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.435365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 459/897] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:44 2022] POST /identity/v3/services => generated 241 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:44.436564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f39541-eec9-44e9-8ecd-660759e41501 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.443406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e939c83-8669-4f27-a7d6-d901e56b2130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.445106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e939c83-8669-4f27-a7d6-d901e56b2130 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-5e939c83-8669-4f27-a7d6-d901e56b2130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:44.463697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:44.464871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:44.465786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:44.465786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:44.465786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:44.470965 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 439/898] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:44.471963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e939c83-8669-4f27-a7d6-d901e56b2130 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.492147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:44.586786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f39541-eec9-44e9-8ecd-660759e41501 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.621120 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e939c83-8669-4f27-a7d6-d901e56b2130 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.626905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b989e3-cda9-459b-aa92-e61bac508232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.627816 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b989e3-cda9-459b-aa92-e61bac508232 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.628307 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b989e3-cda9-459b-aa92-e61bac508232 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.628886 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b989e3-cda9-459b-aa92-e61bac508232 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.629748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b989e3-cda9-459b-aa92-e61bac508232 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.631622 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b989e3-cda9-459b-aa92-e61bac508232 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-f7b989e3-cda9-459b-aa92-e61bac508232 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:44.642691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:44.643690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:44.644582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:44.646140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 460/899] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:44.653041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b989e3-cda9-459b-aa92-e61bac508232 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.663684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366943d-79dc-4f0b-b9d8-03d056b08cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.664303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366943d-79dc-4f0b-b9d8-03d056b08cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.664586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366943d-79dc-4f0b-b9d8-03d056b08cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.664910 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366943d-79dc-4f0b-b9d8-03d056b08cf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.665564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366943d-79dc-4f0b-b9d8-03d056b08cf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:44.665869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366943d-79dc-4f0b-b9d8-03d056b08cf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.667546 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366943d-79dc-4f0b-b9d8-03d056b08cf1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.674863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:44.680356 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 440/900] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:44.686373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366943d-79dc-4f0b-b9d8-03d056b08cf1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.769713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b989e3-cda9-459b-aa92-e61bac508232 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.772378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:44.801547 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366943d-79dc-4f0b-b9d8-03d056b08cf1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.805603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180ac6c-1397-40ee-9a72-c35d13d9741a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.806221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180ac6c-1397-40ee-9a72-c35d13d9741a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.806501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180ac6c-1397-40ee-9a72-c35d13d9741a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.806774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180ac6c-1397-40ee-9a72-c35d13d9741a None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.807531 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180ac6c-1397-40ee-9a72-c35d13d9741a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:44.808044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180ac6c-1397-40ee-9a72-c35d13d9741a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.812695 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180ac6c-1397-40ee-9a72-c35d13d9741a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.819480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 461/901] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:44.820612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180ac6c-1397-40ee-9a72-c35d13d9741a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.842810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89674bd-3222-4d6e-a9ae-b237a1a20ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.843837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89674bd-3222-4d6e-a9ae-b237a1a20ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.844327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89674bd-3222-4d6e-a9ae-b237a1a20ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.844829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89674bd-3222-4d6e-a9ae-b237a1a20ee0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.852551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89674bd-3222-4d6e-a9ae-b237a1a20ee0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.854283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89674bd-3222-4d6e-a9ae-b237a1a20ee0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c89674bd-3222-4d6e-a9ae-b237a1a20ee0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:44.868686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:44.870314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:44.871458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:44.871458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:44.871458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:44.871458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:44.871458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:44.871458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:44.871458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:44.871458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:44.871458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:44.871458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:44.874375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 441/902] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:44.879533 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89674bd-3222-4d6e-a9ae-b237a1a20ee0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.912223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:44.935577 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180ac6c-1397-40ee-9a72-c35d13d9741a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.984122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e475a60-7a4a-4a0a-8b00-a4013b3c2a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.984891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e475a60-7a4a-4a0a-8b00-a4013b3c2a6a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.985270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e475a60-7a4a-4a0a-8b00-a4013b3c2a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.985641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e475a60-7a4a-4a0a-8b00-a4013b3c2a6a None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.986498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e475a60-7a4a-4a0a-8b00-a4013b3c2a6a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:44.986900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e475a60-7a4a-4a0a-8b00-a4013b3c2a6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.988404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e475a60-7a4a-4a0a-8b00-a4013b3c2a6a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.999364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 462/903] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:45.000601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e475a60-7a4a-4a0a-8b00-a4013b3c2a6a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.003323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89674bd-3222-4d6e-a9ae-b237a1a20ee0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:45.045796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7fc00-13fc-499d-8e93-69e2dff862a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:45.047031 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7fc00-13fc-499d-8e93-69e2dff862a1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.047452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7fc00-13fc-499d-8e93-69e2dff862a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.047891 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7fc00-13fc-499d-8e93-69e2dff862a1 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.049159 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7fc00-13fc-499d-8e93-69e2dff862a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:45.049609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7fc00-13fc-499d-8e93-69e2dff862a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:45.051807 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7fc00-13fc-499d-8e93-69e2dff862a1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:45.054303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:45.073138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 442/904] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:45.078674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7fc00-13fc-499d-8e93-69e2dff862a1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.131301 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e475a60-7a4a-4a0a-8b00-a4013b3c2a6a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:45.177646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d331ce-2fc9-4ee9-802c-d0b40baf2028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:45.178897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d331ce-2fc9-4ee9-802c-d0b40baf2028 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.179625 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d331ce-2fc9-4ee9-802c-d0b40baf2028 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.180125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d331ce-2fc9-4ee9-802c-d0b40baf2028 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.181720 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d331ce-2fc9-4ee9-802c-d0b40baf2028 None admin] RBAC: Adding query filter params (interface=public, service_id=2efb846a9baf4e52b25f61e6ad7ab009, region_id=RegionOne) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:45.182241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d331ce-2fc9-4ee9-802c-d0b40baf2028 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:45.184673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d331ce-2fc9-4ee9-802c-d0b40baf2028 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:45.189986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7fc00-13fc-499d-8e93-69e2dff862a1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:45.191947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 463/905] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/endpoints?service_id=2efb846a9baf4e52b25f61e6ad7ab009&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:45.201266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d331ce-2fc9-4ee9-802c-d0b40baf2028 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.210608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:45.237764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75c937-188e-4fb9-8581-a684479f95b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:45.239346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75c937-188e-4fb9-8581-a684479f95b1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.239795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75c937-188e-4fb9-8581-a684479f95b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.240210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75c937-188e-4fb9-8581-a684479f95b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.306507 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d331ce-2fc9-4ee9-802c-d0b40baf2028 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:45.309173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:45.319345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75c937-188e-4fb9-8581-a684479f95b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:45.321024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75c937-188e-4fb9-8581-a684479f95b1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:45.344478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0086563-f6d2-41a7-ba18-dbcb47e43e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:45.345127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0086563-f6d2-41a7-ba18-dbcb47e43e5e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.345407 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0086563-f6d2-41a7-ba18-dbcb47e43e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.345679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0086563-f6d2-41a7-ba18-dbcb47e43e5e None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.346291 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0086563-f6d2-41a7-ba18-dbcb47e43e5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:45.347678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0086563-f6d2-41a7-ba18-dbcb47e43e5e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:45.359727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 464/906] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:45 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:20:45.439186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 443/907] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:20:45.445280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0086563-f6d2-41a7-ba18-dbcb47e43e5e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.530863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:45.565428 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0086563-f6d2-41a7-ba18-dbcb47e43e5e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:45.581932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be39ae-912e-4d4a-99c7-4eeacad6fb8e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.582301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be39ae-912e-4d4a-99c7-4eeacad6fb8e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.582650 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be39ae-912e-4d4a-99c7-4eeacad6fb8e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.585376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 444/908] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:45.590137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055190d9-c3ae-4153-a7b5-f88a3b1dd745 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.590491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055190d9-c3ae-4153-a7b5-f88a3b1dd745 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.590831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055190d9-c3ae-4153-a7b5-f88a3b1dd745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.607086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114df19-1634-40e0-b263-720a44844b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:45.607995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114df19-1634-40e0-b263-720a44844b20 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.608468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114df19-1634-40e0-b263-720a44844b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.608999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114df19-1634-40e0-b263-720a44844b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.609632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-055190d9-c3ae-4153-a7b5-f88a3b1dd745 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:45.611073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-055190d9-c3ae-4153-a7b5-f88a3b1dd745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:45.623459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114df19-1634-40e0-b263-720a44844b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:45.625770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114df19-1634-40e0-b263-720a44844b20 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-0114df19-1634-40e0-b263-720a44844b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:45.645698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:45.646666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:45.647536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:45.647536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:45.647536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:45.650284 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 465/909] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:45.656258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114df19-1634-40e0-b263-720a44844b20 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.693166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 445/910] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:45.764038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114df19-1634-40e0-b263-720a44844b20 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:45.787969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:45.799415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6a63a-3f23-4009-90ca-10e96b9baa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:45.800078 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6a63a-3f23-4009-90ca-10e96b9baa21 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.800357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6a63a-3f23-4009-90ca-10e96b9baa21 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.800700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6a63a-3f23-4009-90ca-10e96b9baa21 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.801355 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6a63a-3f23-4009-90ca-10e96b9baa21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:45.801703 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6a63a-3f23-4009-90ca-10e96b9baa21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:45.803086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6a63a-3f23-4009-90ca-10e96b9baa21 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:45.815990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 466/911] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:45.821589 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055190d9-c3ae-4153-a7b5-f88a3b1dd745 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.847813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:45.929005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055190d9-c3ae-4153-a7b5-f88a3b1dd745 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:45.970297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491eaf6b-6db3-470c-b65a-2330cc81c0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:45.970933 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491eaf6b-6db3-470c-b65a-2330cc81c0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.971208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491eaf6b-6db3-470c-b65a-2330cc81c0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.971478 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491eaf6b-6db3-470c-b65a-2330cc81c0b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.978680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491eaf6b-6db3-470c-b65a-2330cc81c0b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:45.980447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491eaf6b-6db3-470c-b65a-2330cc81c0b8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:45.984216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6a63a-3f23-4009-90ca-10e96b9baa21 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-491eaf6b-6db3-470c-b65a-2330cc81c0b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:45.992461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:45.993406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:45.997529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:45.997529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:45.997529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:45.997529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:45.997529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:45.997529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:45.997529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:45.997529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:45.997529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 446/912] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:46.001696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491eaf6b-6db3-470c-b65a-2330cc81c0b8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:46.098955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:46.120345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491eaf6b-6db3-470c-b65a-2330cc81c0b8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.127623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6a63a-3f23-4009-90ca-10e96b9baa21 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.162665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cde03f-92f1-41a5-b3ae-4e52752a8f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.163272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cde03f-92f1-41a5-b3ae-4e52752a8f46 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.163547 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cde03f-92f1-41a5-b3ae-4e52752a8f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.163813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cde03f-92f1-41a5-b3ae-4e52752a8f46 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.164513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cde03f-92f1-41a5-b3ae-4e52752a8f46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:46.164844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cde03f-92f1-41a5-b3ae-4e52752a8f46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:46.166299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cde03f-92f1-41a5-b3ae-4e52752a8f46 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:46.167317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7268ad-e872-42cd-ad72-c57c695289db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.168188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7268ad-e872-42cd-ad72-c57c695289db None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.168844 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7268ad-e872-42cd-ad72-c57c695289db None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.169354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7268ad-e872-42cd-ad72-c57c695289db None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.170217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7268ad-e872-42cd-ad72-c57c695289db None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:46.171793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7268ad-e872-42cd-ad72-c57c695289db None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:46.184755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 447/913] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-bb7268ad-e872-42cd-ad72-c57c695289db None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:46.184997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:46.185879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:46.186718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 467/914] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:46.191958 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cde03f-92f1-41a5-b3ae-4e52752a8f46 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:46.193460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7268ad-e872-42cd-ad72-c57c695289db None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:46.247313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:46.305263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7268ad-e872-42cd-ad72-c57c695289db None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.316084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cde03f-92f1-41a5-b3ae-4e52752a8f46 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.347900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff3653-e46b-4aa6-bf46-b411d1d2ec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.348516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff3653-e46b-4aa6-bf46-b411d1d2ec00 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.348934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff3653-e46b-4aa6-bf46-b411d1d2ec00 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.349281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff3653-e46b-4aa6-bf46-b411d1d2ec00 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.349996 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff3653-e46b-4aa6-bf46-b411d1d2ec00 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:46.350298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff3653-e46b-4aa6-bf46-b411d1d2ec00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:46.352330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff3653-e46b-4aa6-bf46-b411d1d2ec00 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:46.360560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024c28c-2779-4a38-825b-427281001b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.361576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024c28c-2779-4a38-825b-427281001b26 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.362069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024c28c-2779-4a38-825b-427281001b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.362550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024c28c-2779-4a38-825b-427281001b26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.363168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 468/915] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:46.363933 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024c28c-2779-4a38-825b-427281001b26 None admin] RBAC: Adding query filter params (role.id=1b4412853cfe43d3b524d26be06750ef, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=34a3cadee8ea4b3e9dff0bc23dfe9dc5) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:46.364468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff3653-e46b-4aa6-bf46-b411d1d2ec00 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:46.364575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024c28c-2779-4a38-825b-427281001b26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:46.367629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024c28c-2779-4a38-825b-427281001b26 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:46.380823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 448/916] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:46 2022] GET /identity/v3/role_assignments?user.id=34a3cadee8ea4b3e9dff0bc23dfe9dc5&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=1b4412853cfe43d3b524d26be06750ef => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:46.397345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ec2c3-9d9a-4eec-bf2f-e6687ba790db None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.397665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ec2c3-9d9a-4eec-bf2f-e6687ba790db None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.397900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ec2c3-9d9a-4eec-bf2f-e6687ba790db None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.404138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 449/917] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:46.405611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f6d82-1e3e-43c6-ba6b-3f08f8c7a41c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.405896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f6d82-1e3e-43c6-ba6b-3f08f8c7a41c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.406170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f6d82-1e3e-43c6-ba6b-3f08f8c7a41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.420193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:46.427143 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-648f6d82-1e3e-43c6-ba6b-3f08f8c7a41c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:46.428460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-648f6d82-1e3e-43c6-ba6b-3f08f8c7a41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:46.501214 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff3653-e46b-4aa6-bf46-b411d1d2ec00 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.520150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:46.523007 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 450/918] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:46.546315 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f959a-a01d-4c87-86e6-1e4493ff73e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.547329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f959a-a01d-4c87-86e6-1e4493ff73e4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.547805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f959a-a01d-4c87-86e6-1e4493ff73e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.548251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f959a-a01d-4c87-86e6-1e4493ff73e4 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.549115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f959a-a01d-4c87-86e6-1e4493ff73e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:46.549594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f959a-a01d-4c87-86e6-1e4493ff73e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:46.551194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f959a-a01d-4c87-86e6-1e4493ff73e4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:46.557491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 469/919] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:46.565504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f6d82-1e3e-43c6-ba6b-3f08f8c7a41c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:46.673353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f6d82-1e3e-43c6-ba6b-3f08f8c7a41c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.710980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa61f9-d574-4b8f-8c4e-bdc5bb08d8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.711929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa61f9-d574-4b8f-8c4e-bdc5bb08d8d9 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.712781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa61f9-d574-4b8f-8c4e-bdc5bb08d8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.713226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa61f9-d574-4b8f-8c4e-bdc5bb08d8d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.714208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa61f9-d574-4b8f-8c4e-bdc5bb08d8d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:46.715774 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa61f9-d574-4b8f-8c4e-bdc5bb08d8d9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:46.721694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:46.739758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 451/920] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:46.791265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f959a-a01d-4c87-86e6-1e4493ff73e4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:46.905982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f959a-a01d-4c87-86e6-1e4493ff73e4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.951173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6910734-5696-434d-8078-3027538da027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.952162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6910734-5696-434d-8078-3027538da027 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.952722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6910734-5696-434d-8078-3027538da027 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.953189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6910734-5696-434d-8078-3027538da027 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.953961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6910734-5696-434d-8078-3027538da027 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:46.955534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6910734-5696-434d-8078-3027538da027 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-a6910734-5696-434d-8078-3027538da027 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:46.970115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:46.971148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:46.972502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:46.976540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 470/921] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:46.986630 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa61f9-d574-4b8f-8c4e-bdc5bb08d8d9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:47.101545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa61f9-d574-4b8f-8c4e-bdc5bb08d8d9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:47.138144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636b938-335d-442a-9324-2ce3673cc0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:47.138758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636b938-335d-442a-9324-2ce3673cc0af None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.139035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636b938-335d-442a-9324-2ce3673cc0af None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.139307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636b938-335d-442a-9324-2ce3673cc0af None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.139945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636b938-335d-442a-9324-2ce3673cc0af None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:47.140245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636b938-335d-442a-9324-2ce3673cc0af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:47.141672 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636b938-335d-442a-9324-2ce3673cc0af None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:47.146982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 452/922] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:47.152751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6910734-5696-434d-8078-3027538da027 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:47.191537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:47.262696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6910734-5696-434d-8078-3027538da027 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:47.298564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393b1f1-cf01-480f-9e31-2802ae3efcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:47.299548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393b1f1-cf01-480f-9e31-2802ae3efcba None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.300024 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393b1f1-cf01-480f-9e31-2802ae3efcba None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.300509 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393b1f1-cf01-480f-9e31-2802ae3efcba None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.301360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393b1f1-cf01-480f-9e31-2802ae3efcba None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:47.301852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393b1f1-cf01-480f-9e31-2802ae3efcba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:47.303415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393b1f1-cf01-480f-9e31-2802ae3efcba None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:47.309072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 471/923] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:47.315274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636b938-335d-442a-9324-2ce3673cc0af None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:47.380789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:47.428144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636b938-335d-442a-9324-2ce3673cc0af None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:47.463669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2c786-0a10-4c3d-9187-53daec43d638 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.464299 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2c786-0a10-4c3d-9187-53daec43d638 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.464793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2c786-0a10-4c3d-9187-53daec43d638 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.466794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 472/924] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:47.466901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec598a-ae37-476e-b4d6-cc9c02e17ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:47.471057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec598a-ae37-476e-b4d6-cc9c02e17ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.471057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec598a-ae37-476e-b4d6-cc9c02e17ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.471057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec598a-ae37-476e-b4d6-cc9c02e17ba5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.471057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec598a-ae37-476e-b4d6-cc9c02e17ba5 None admin] RBAC: Adding query filter params (interface=public, service_id=8acea1b94fa94e6e98025f17920f0b50, region_id=RegionOne) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:47.471057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec598a-ae37-476e-b4d6-cc9c02e17ba5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:47.471057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec598a-ae37-476e-b4d6-cc9c02e17ba5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:47.472880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0a969-be39-4933-8f45-ed4af20885f0 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.473361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0a969-be39-4933-8f45-ed4af20885f0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.473852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0a969-be39-4933-8f45-ed4af20885f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.479746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 453/925] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:47 2022] GET /identity/v3/endpoints?service_id=8acea1b94fa94e6e98025f17920f0b50&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:47.484182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec598a-ae37-476e-b4d6-cc9c02e17ba5 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:47.494280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7cf0a969-be39-4933-8f45-ed4af20885f0 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:47.495711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7cf0a969-be39-4933-8f45-ed4af20885f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:47.591897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec598a-ae37-476e-b4d6-cc9c02e17ba5 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:47.596851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 473/926] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:47.615736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:47.631226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2dd71-fc53-457b-8cc6-358fbe69d373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:47.632059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2dd71-fc53-457b-8cc6-358fbe69d373 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.632541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2dd71-fc53-457b-8cc6-358fbe69d373 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.633005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2dd71-fc53-457b-8cc6-358fbe69d373 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.633788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2dd71-fc53-457b-8cc6-358fbe69d373 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:47.635406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2dd71-fc53-457b-8cc6-358fbe69d373 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:47.641896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 454/927] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:47 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:20:47.870414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0a969-be39-4933-8f45-ed4af20885f0 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:47.902020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:47.984524 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0a969-be39-4933-8f45-ed4af20885f0 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:48.019941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc841d-6449-4d28-afc8-fabac42c64eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:48.020573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc841d-6449-4d28-afc8-fabac42c64eb None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.020875 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc841d-6449-4d28-afc8-fabac42c64eb None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.021145 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc841d-6449-4d28-afc8-fabac42c64eb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.021725 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc841d-6449-4d28-afc8-fabac42c64eb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:48.023117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc841d-6449-4d28-afc8-fabac42c64eb None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-f3cc841d-6449-4d28-afc8-fabac42c64eb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:48.031843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:48.032921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:48.033916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 474/928] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Aug 24 12:20:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:48.041279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2dd71-fc53-457b-8cc6-358fbe69d373 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:48.147475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2dd71-fc53-457b-8cc6-358fbe69d373 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:48.183707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3673e-23a7-46bd-8815-82739f0cafb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:48.184749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3673e-23a7-46bd-8815-82739f0cafb5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.185240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3673e-23a7-46bd-8815-82739f0cafb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.185713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3673e-23a7-46bd-8815-82739f0cafb5 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.186561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3673e-23a7-46bd-8815-82739f0cafb5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:48.187103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3673e-23a7-46bd-8815-82739f0cafb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:48.188715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3673e-23a7-46bd-8815-82739f0cafb5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:48.194379 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 455/929] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:48.200697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc841d-6449-4d28-afc8-fabac42c64eb None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:48.301298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:48.310080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc841d-6449-4d28-afc8-fabac42c64eb None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:48.348223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a69487-90a6-4365-8733-e4f54cbb2669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:48.349276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69487-90a6-4365-8733-e4f54cbb2669 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.349770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69487-90a6-4365-8733-e4f54cbb2669 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.350247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69487-90a6-4365-8733-e4f54cbb2669 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.351143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a69487-90a6-4365-8733-e4f54cbb2669 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:48.351712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a69487-90a6-4365-8733-e4f54cbb2669 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:48.353374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a69487-90a6-4365-8733-e4f54cbb2669 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:48.359289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 475/930] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:48.363932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fea446-d4e9-4e03-9f80-1886dda0d316 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.364244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fea446-d4e9-4e03-9f80-1886dda0d316 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.364538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fea446-d4e9-4e03-9f80-1886dda0d316 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.365634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a69487-90a6-4365-8733-e4f54cbb2669 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:48.366297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 456/931] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:48.371959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86acbc-92a7-448f-8b9b-66ccfe06da85 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.372235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86acbc-92a7-448f-8b9b-66ccfe06da85 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.372506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86acbc-92a7-448f-8b9b-66ccfe06da85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.391298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6b86acbc-92a7-448f-8b9b-66ccfe06da85 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:48.392957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6b86acbc-92a7-448f-8b9b-66ccfe06da85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:48.454229 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:48.477576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a69487-90a6-4365-8733-e4f54cbb2669 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:48.496319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 457/932] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:48.514170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870754e-e9d9-45fc-982c-f95a3cc41a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:48.515129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870754e-e9d9-45fc-982c-f95a3cc41a2b None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.515604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870754e-e9d9-45fc-982c-f95a3cc41a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.516071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870754e-e9d9-45fc-982c-f95a3cc41a2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.516974 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870754e-e9d9-45fc-982c-f95a3cc41a2b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:48.518572 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870754e-e9d9-45fc-982c-f95a3cc41a2b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:48.541057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 476/933] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:48.770816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86acbc-92a7-448f-8b9b-66ccfe06da85 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:48.837624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddc84a-0aaf-4387-b4bf-663693ef2713 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.838365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddc84a-0aaf-4387-b4bf-663693ef2713 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.838698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddc84a-0aaf-4387-b4bf-663693ef2713 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.841646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 477/934] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:48.845946 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87c6d1-8e92-4703-b818-ed1a7d243a8c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.846476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87c6d1-8e92-4703-b818-ed1a7d243a8c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.846962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87c6d1-8e92-4703-b818-ed1a7d243a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.865975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2d87c6d1-8e92-4703-b818-ed1a7d243a8c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:48.867559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2d87c6d1-8e92-4703-b818-ed1a7d243a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:48.878188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:48.884328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86acbc-92a7-448f-8b9b-66ccfe06da85 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:48.937088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e49ead-e676-4c55-b45e-fe327cc4a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:48.937957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e49ead-e676-4c55-b45e-fe327cc4a2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.938229 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e49ead-e676-4c55-b45e-fe327cc4a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.938494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e49ead-e676-4c55-b45e-fe327cc4a2f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.948712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 478/935] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:48.954067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e49ead-e676-4c55-b45e-fe327cc4a2f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-30e49ead-e676-4c55-b45e-fe327cc4a2f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:48.958863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:48.959959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:48.960834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:48.960834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:48.960834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:48.960834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:48.960834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:48.960834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:48.960834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:48.960834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:48.960834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:48.960834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:48.960834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:48.960834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:48.963484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 458/936] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 24 12:20:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:48.969784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87c6d1-8e92-4703-b818-ed1a7d243a8c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:49.076969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87c6d1-8e92-4703-b818-ed1a7d243a8c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:49.116471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39618b7-e6d0-4a9f-afca-ee112c98f191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:49.117560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39618b7-e6d0-4a9f-afca-ee112c98f191 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.118041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39618b7-e6d0-4a9f-afca-ee112c98f191 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.118506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39618b7-e6d0-4a9f-afca-ee112c98f191 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.119379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39618b7-e6d0-4a9f-afca-ee112c98f191 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:49.120172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39618b7-e6d0-4a9f-afca-ee112c98f191 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:49.121768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39618b7-e6d0-4a9f-afca-ee112c98f191 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:49.131008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 479/937] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 12:20:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:49.136824 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e49ead-e676-4c55-b45e-fe327cc4a2f6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:49.172272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:49.241271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e49ead-e676-4c55-b45e-fe327cc4a2f6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:49.254144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39618b7-e6d0-4a9f-afca-ee112c98f191 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:49.265559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:49.276885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77855d-9957-47d1-91c1-97555726b6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:49.277834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77855d-9957-47d1-91c1-97555726b6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.278293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77855d-9957-47d1-91c1-97555726b6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.278827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77855d-9957-47d1-91c1-97555726b6a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.289908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77855d-9957-47d1-91c1-97555726b6a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:49.292280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77855d-9957-47d1-91c1-97555726b6a6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3f77855d-9957-47d1-91c1-97555726b6a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:49.311271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:49.312285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:49.313210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:49.313210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:49.313210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:49.320220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 459/938] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:49.322055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77855d-9957-47d1-91c1-97555726b6a6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:49.384680 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39618b7-e6d0-4a9f-afca-ee112c98f191 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:49.417669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751408-b31b-4619-93f0-5990fdf4ce5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:49.418480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751408-b31b-4619-93f0-5990fdf4ce5a None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.418851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751408-b31b-4619-93f0-5990fdf4ce5a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.419212 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751408-b31b-4619-93f0-5990fdf4ce5a None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.420069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b751408-b31b-4619-93f0-5990fdf4ce5a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:49.421715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b751408-b31b-4619-93f0-5990fdf4ce5a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:49.424641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: WARNING keystone.server.flask.common [None req-0b751408-b31b-4619-93f0-5990fdf4ce5a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 12:20:49.430891 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:49.446132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.sql.core [None req-0b751408-b31b-4619-93f0-5990fdf4ce5a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 12:20:49.446132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 12:20:49.446132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [parameters: {'id': '9e3d7ede035f49468cc1a229c996fb31', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 12:20:49.446132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81635) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 12:20:49.447696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77855d-9957-47d1-91c1-97555726b6a6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-0b751408-b31b-4619-93f0-5990fdf4ce5a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9e3d7ede035f49468cc1a229c996fb31.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9e3d7ede035f49468cc1a229c996fb31. Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:20:49.467715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application result.read() Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 12:20:49.468588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application yield resource Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise self.value Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application session.commit() Aug 24 12:20:49.469591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.session.flush() Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application compat.raise_( Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 12:20:49.470487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 12:20:49.471392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application result.read() Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:20:49.472207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [parameters: {'id': '9e3d7ede035f49468cc1a229c996fb31', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:49.473776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:49.474768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:49.475943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:49.475943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 12:20:49.475943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 12:20:49.475943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:49.475943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:49.475943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 12:20:49.475943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 12:20:49.475943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9e3d7ede035f49468cc1a229c996fb31. Aug 24 12:20:49.475943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:49.483403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 480/939] 10.222.0.27 () {60 vars in 1140 bytes} [Wed Aug 24 12:20:49 2022] POST /identity/v3/projects => generated 262 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:49.484797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751408-b31b-4619-93f0-5990fdf4ce5a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:49.497414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e05993-60e0-450d-aa86-0a37e92ed1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:49.498270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e05993-60e0-450d-aa86-0a37e92ed1cf None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.498742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e05993-60e0-450d-aa86-0a37e92ed1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.499217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e05993-60e0-450d-aa86-0a37e92ed1cf None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.500062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e05993-60e0-450d-aa86-0a37e92ed1cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:49.500495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e05993-60e0-450d-aa86-0a37e92ed1cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:49.502205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e05993-60e0-450d-aa86-0a37e92ed1cf None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:49.516287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 460/940] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:49.524809 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e05993-60e0-450d-aa86-0a37e92ed1cf None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:49.557316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:49.594811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751408-b31b-4619-93f0-5990fdf4ce5a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:49.630660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbff93-71c7-4164-b82d-b6eaf9fee065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:49.631446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbff93-71c7-4164-b82d-b6eaf9fee065 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.631823 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbff93-71c7-4164-b82d-b6eaf9fee065 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.632207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbff93-71c7-4164-b82d-b6eaf9fee065 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.640250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbff93-71c7-4164-b82d-b6eaf9fee065 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:49.642279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbff93-71c7-4164-b82d-b6eaf9fee065 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-efcbff93-71c7-4164-b82d-b6eaf9fee065 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:49.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:49.657924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:49.659017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:49.659017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:49.659017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:49.659017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:49.659017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:49.659017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:49.659017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:49.659017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:49.660639 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 481/941] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:49.664062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e05993-60e0-450d-aa86-0a37e92ed1cf None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:49.666919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbff93-71c7-4164-b82d-b6eaf9fee065 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:49.706118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b6aac-b818-4e54-9424-175bb8978c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:49.706924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b6aac-b818-4e54-9424-175bb8978c3d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.707294 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b6aac-b818-4e54-9424-175bb8978c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.707664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b6aac-b818-4e54-9424-175bb8978c3d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.717073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b6aac-b818-4e54-9424-175bb8978c3d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:49.719286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b6aac-b818-4e54-9424-175bb8978c3d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-970b6aac-b818-4e54-9424-175bb8978c3d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:49.734586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:49.735792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:49.737012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:49.737012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:49.737012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:49.737012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:49.737012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:49.737012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:49.737012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:49.737012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:49.737012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:49.737012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:49.737639 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 461/942] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 12:20:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:49.738749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44ee52-b5c8-4165-ad32-fde6c648ac1c None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.739056 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44ee52-b5c8-4165-ad32-fde6c648ac1c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.739328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44ee52-b5c8-4165-ad32-fde6c648ac1c None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.744282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 462/943] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:49.745498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3976171-bd7d-4395-8af1-d4c6b514904b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.745775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3976171-bd7d-4395-8af1-d4c6b514904b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.746091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3976171-bd7d-4395-8af1-d4c6b514904b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.759210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:49.765832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a3976171-bd7d-4395-8af1-d4c6b514904b None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:49.767543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a3976171-bd7d-4395-8af1-d4c6b514904b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:49.794827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbff93-71c7-4164-b82d-b6eaf9fee065 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:49.817741 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:49.831461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3e152-fd56-466c-8b94-1d2b027ff0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:49.831972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3e152-fd56-466c-8b94-1d2b027ff0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.832249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3e152-fd56-466c-8b94-1d2b027ff0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.832682 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3e152-fd56-466c-8b94-1d2b027ff0a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.833286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3e152-fd56-466c-8b94-1d2b027ff0a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:49.833569 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3e152-fd56-466c-8b94-1d2b027ff0a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:49.835028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3e152-fd56-466c-8b94-1d2b027ff0a5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:49.841156 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 463/944] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:49.841156 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3976171-bd7d-4395-8af1-d4c6b514904b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:49.862785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 482/945] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:49.958765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3976171-bd7d-4395-8af1-d4c6b514904b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:49.998782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ef529-602a-495f-a7d6-4e154d77f45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:49.999411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ef529-602a-495f-a7d6-4e154d77f45d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.999685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ef529-602a-495f-a7d6-4e154d77f45d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.999954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ef529-602a-495f-a7d6-4e154d77f45d None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.000729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ef529-602a-495f-a7d6-4e154d77f45d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:50.001037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ef529-602a-495f-a7d6-4e154d77f45d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.002585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ef529-602a-495f-a7d6-4e154d77f45d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.005749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:50.022121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 464/946] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 24 12:20:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:50.028849 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3e152-fd56-466c-8b94-1d2b027ff0a5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.121384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ef529-602a-495f-a7d6-4e154d77f45d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.143163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3e152-fd56-466c-8b94-1d2b027ff0a5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.191168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173445ff-f631-4d84-bcb9-d80bcb0f1d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.192397 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173445ff-f631-4d84-bcb9-d80bcb0f1d85 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.192879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173445ff-f631-4d84-bcb9-d80bcb0f1d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.193699 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173445ff-f631-4d84-bcb9-d80bcb0f1d85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.208585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173445ff-f631-4d84-bcb9-d80bcb0f1d85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.210906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173445ff-f631-4d84-bcb9-d80bcb0f1d85 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-173445ff-f631-4d84-bcb9-d80bcb0f1d85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:50.230228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:50.231277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:50.232177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:50.232177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:50.232177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:50.236324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 483/947] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:50.237854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173445ff-f631-4d84-bcb9-d80bcb0f1d85 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.245226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ef529-602a-495f-a7d6-4e154d77f45d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.274057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:50.292143 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce7fda-dc3f-401d-86ef-19e7568c2960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.293449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce7fda-dc3f-401d-86ef-19e7568c2960 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.293939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce7fda-dc3f-401d-86ef-19e7568c2960 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.294414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce7fda-dc3f-401d-86ef-19e7568c2960 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.295649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce7fda-dc3f-401d-86ef-19e7568c2960 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.297426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce7fda-dc3f-401d-86ef-19e7568c2960 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-f1ce7fda-dc3f-401d-86ef-19e7568c2960 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:50.307268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:50.309041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:50.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 465/948] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:50.316870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce7fda-dc3f-401d-86ef-19e7568c2960 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.357665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173445ff-f631-4d84-bcb9-d80bcb0f1d85 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.394513 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674874fb-1c12-4b71-9b5a-aafa39a5b284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.395281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674874fb-1c12-4b71-9b5a-aafa39a5b284 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.395653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674874fb-1c12-4b71-9b5a-aafa39a5b284 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.396068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674874fb-1c12-4b71-9b5a-aafa39a5b284 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.396847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674874fb-1c12-4b71-9b5a-aafa39a5b284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:50.397244 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674874fb-1c12-4b71-9b5a-aafa39a5b284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.398708 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674874fb-1c12-4b71-9b5a-aafa39a5b284 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.402304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:50.417359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 484/949] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:50.418416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674874fb-1c12-4b71-9b5a-aafa39a5b284 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.452312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce7fda-dc3f-401d-86ef-19e7568c2960 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.491592 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f2948-cac0-4bcc-95c9-762f9c29cc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.492137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f2948-cac0-4bcc-95c9-762f9c29cc32 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.492398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f2948-cac0-4bcc-95c9-762f9c29cc32 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.492690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f2948-cac0-4bcc-95c9-762f9c29cc32 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.493336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f2948-cac0-4bcc-95c9-762f9c29cc32 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:50.493638 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f2948-cac0-4bcc-95c9-762f9c29cc32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.495321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f2948-cac0-4bcc-95c9-762f9c29cc32 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.504538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 466/950] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:50.504538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dacf0-7165-4567-b8db-a2366dd1177e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.504538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dacf0-7165-4567-b8db-a2366dd1177e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.504538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dacf0-7165-4567-b8db-a2366dd1177e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.504538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 467/951] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:50.507518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dacf0-7165-4567-b8db-a2366dd1177e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.534614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:50.543700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674874fb-1c12-4b71-9b5a-aafa39a5b284 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.584340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e14bcd-b3e6-46fe-9c70-d78c69b74846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.585000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e14bcd-b3e6-46fe-9c70-d78c69b74846 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.585275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e14bcd-b3e6-46fe-9c70-d78c69b74846 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.585542 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e14bcd-b3e6-46fe-9c70-d78c69b74846 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.593046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e14bcd-b3e6-46fe-9c70-d78c69b74846 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.594505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e14bcd-b3e6-46fe-9c70-d78c69b74846 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-01e14bcd-b3e6-46fe-9c70-d78c69b74846 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:50.605786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:50.606709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:50.608236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:50.608236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:50.608236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:50.608236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:50.608236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:50.608236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:50.608236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:50.608236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:50.608236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 485/952] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:50.608943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2d033-ef35-4f46-a03a-4f6545ec511c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.608943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2d033-ef35-4f46-a03a-4f6545ec511c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.609084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2d033-ef35-4f46-a03a-4f6545ec511c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.620411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dacf0-7165-4567-b8db-a2366dd1177e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.628876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-fdc2d033-ef35-4f46-a03a-4f6545ec511c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:50.630408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fdc2d033-ef35-4f46-a03a-4f6545ec511c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:50.644403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:50.659297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8f883-7f9a-4fd0-b7d4-7c6ba6fcb2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.660844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8f883-7f9a-4fd0-b7d4-7c6ba6fcb2ea None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.661325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8f883-7f9a-4fd0-b7d4-7c6ba6fcb2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.662064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8f883-7f9a-4fd0-b7d4-7c6ba6fcb2ea None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.663306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8f883-7f9a-4fd0-b7d4-7c6ba6fcb2ea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:50.664021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8f883-7f9a-4fd0-b7d4-7c6ba6fcb2ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.665617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8f883-7f9a-4fd0-b7d4-7c6ba6fcb2ea None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.671628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 468/953] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:50.672219 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8f883-7f9a-4fd0-b7d4-7c6ba6fcb2ea None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.703380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 486/954] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:50.710069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2d033-ef35-4f46-a03a-4f6545ec511c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.789993 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8f883-7f9a-4fd0-b7d4-7c6ba6fcb2ea None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.816444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:50.834495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2d033-ef35-4f46-a03a-4f6545ec511c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.840312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c88cd7-9f55-4b3b-b298-ac3f5ea2de2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.841228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c88cd7-9f55-4b3b-b298-ac3f5ea2de2a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.841735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c88cd7-9f55-4b3b-b298-ac3f5ea2de2a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.842196 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c88cd7-9f55-4b3b-b298-ac3f5ea2de2a None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.843881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c88cd7-9f55-4b3b-b298-ac3f5ea2de2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:50.844408 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c88cd7-9f55-4b3b-b298-ac3f5ea2de2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.846721 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c88cd7-9f55-4b3b-b298-ac3f5ea2de2a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.863831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 469/955] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:50.872713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c88cd7-9f55-4b3b-b298-ac3f5ea2de2a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.876376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225d16d-a23f-416f-81ce-cb76868faae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.877092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225d16d-a23f-416f-81ce-cb76868faae1 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.877454 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225d16d-a23f-416f-81ce-cb76868faae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.877848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225d16d-a23f-416f-81ce-cb76868faae1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.878601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225d16d-a23f-416f-81ce-cb76868faae1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.888376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225d16d-a23f-416f-81ce-cb76868faae1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.921147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 487/956] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:50.928534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:50.974401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225d16d-a23f-416f-81ce-cb76868faae1 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:51.000416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c88cd7-9f55-4b3b-b298-ac3f5ea2de2a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:51.059156 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c60593-3fed-41eb-976c-4b648b6cb999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:51.059971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c60593-3fed-41eb-976c-4b648b6cb999 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:51.060255 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c60593-3fed-41eb-976c-4b648b6cb999 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:51.060528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c60593-3fed-41eb-976c-4b648b6cb999 None admin] PATH_INFO: `/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/34a3cadee8ea4b3e9dff0bc23dfe9dc5/roles/1b4412853cfe43d3b524d26be06750ef` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:51.063840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c60593-3fed-41eb-976c-4b648b6cb999 None admin] RBAC: Authorizing `identity:create_grant(project_id=70a5f9cf0499405d82f5f5a47c2877eb, user_id=34a3cadee8ea4b3e9dff0bc23dfe9dc5, role_id=1b4412853cfe43d3b524d26be06750ef)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:51.065370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c60593-3fed-41eb-976c-4b648b6cb999 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:51.087526 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 470/957] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:50 2022] PUT /identity/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/34a3cadee8ea4b3e9dff0bc23dfe9dc5/roles/1b4412853cfe43d3b524d26be06750ef => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:20:51.104625 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225d16d-a23f-416f-81ce-cb76868faae1 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:51.154476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61a69d-1fc5-4829-a699-20e711877e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:51.155139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61a69d-1fc5-4829-a699-20e711877e12 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:51.155415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61a69d-1fc5-4829-a699-20e711877e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:51.155728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61a69d-1fc5-4829-a699-20e711877e12 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:51.156483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61a69d-1fc5-4829-a699-20e711877e12 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:51.157932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61a69d-1fc5-4829-a699-20e711877e12 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-2b61a69d-1fc5-4829-a699-20e711877e12 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:51.168749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:51.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:51.171163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 488/958] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:51.178329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c60593-3fed-41eb-976c-4b648b6cb999 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:51.299492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c60593-3fed-41eb-976c-4b648b6cb999 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:51.307685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:51.337564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ed119-b50e-493c-9328-098dedcf1ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:51.338283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ed119-b50e-493c-9328-098dedcf1ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:51.338565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ed119-b50e-493c-9328-098dedcf1ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:51.338917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ed119-b50e-493c-9328-098dedcf1ce8 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:51.339563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ed119-b50e-493c-9328-098dedcf1ce8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:51.339919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ed119-b50e-493c-9328-098dedcf1ce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:51.341576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ed119-b50e-493c-9328-098dedcf1ce8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:51.347817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 471/959] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:51.354266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61a69d-1fc5-4829-a699-20e711877e12 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:51.359598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:51.467518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61a69d-1fc5-4829-a699-20e711877e12 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:51.502667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e02d3-e74a-488e-a631-8da575468a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:51.503272 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e02d3-e74a-488e-a631-8da575468a6c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:51.503546 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e02d3-e74a-488e-a631-8da575468a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:51.503811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e02d3-e74a-488e-a631-8da575468a6c None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:51.504438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e02d3-e74a-488e-a631-8da575468a6c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:51.504759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e02d3-e74a-488e-a631-8da575468a6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:51.506130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e02d3-e74a-488e-a631-8da575468a6c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:51.511582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 489/960] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:51.900442 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae0658-27cf-4aaf-8232-8f7a46161c24 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:51.900928 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae0658-27cf-4aaf-8232-8f7a46161c24 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:51.901206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae0658-27cf-4aaf-8232-8f7a46161c24 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:51.902790 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 472/961] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:51.908308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0054f-eb11-4e6f-b9a5-35e850ed67c1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:51.908593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0054f-eb11-4e6f-b9a5-35e850ed67c1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:51.908946 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0054f-eb11-4e6f-b9a5-35e850ed67c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:51.927634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-92a0054f-eb11-4e6f-b9a5-35e850ed67c1 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:51.928847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-92a0054f-eb11-4e6f-b9a5-35e850ed67c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:51.999174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 490/962] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:52.001035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:52.268366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae0658-27cf-4aaf-8232-8f7a46161c24 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:52.370702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae0658-27cf-4aaf-8232-8f7a46161c24 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:52.412927 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ff7d0-2e7f-433b-9b09-40ef9bb59337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:52.413532 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ff7d0-2e7f-433b-9b09-40ef9bb59337 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:52.413804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ff7d0-2e7f-433b-9b09-40ef9bb59337 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:52.414066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ff7d0-2e7f-433b-9b09-40ef9bb59337 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:52.414712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ff7d0-2e7f-433b-9b09-40ef9bb59337 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:52.416138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ff7d0-2e7f-433b-9b09-40ef9bb59337 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:52.418766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: WARNING keystone.server.flask.common [None req-b72ff7d0-2e7f-433b-9b09-40ef9bb59337 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 12:20:52.431999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.sql.core [None req-b72ff7d0-2e7f-433b-9b09-40ef9bb59337 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 12:20:52.431999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 12:20:52.431999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [parameters: {'user_id': '812ec873c0c641dfb4ac912166fc1440', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 12:20:52.431999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81634) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b72ff7d0-2e7f-433b-9b09-40ef9bb59337 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result.read() Aug 24 12:20:52.438693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:20:52.439540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application yield resource Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise self.value Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application session.commit() Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 12:20:52.440960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.session.flush() Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application compat.raise_( Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 12:20:52.442078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:20:52.443820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application result.read() Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:20:52.449267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [parameters: {'user_id': '812ec873c0c641dfb4ac912166fc1440', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:52.450306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 12:20:52.451108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 12:20:52.452000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 12:20:52.452000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 12:20:52.452000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 12:20:52.452000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 12:20:52.452000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 12:20:52.452000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:52.452000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 473/963] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 24 12:20:52 2022] POST /identity/v3/users => generated 160 bytes in 172 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:52.453930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0054f-eb11-4e6f-b9a5-35e850ed67c1 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:52.559072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0054f-eb11-4e6f-b9a5-35e850ed67c1 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:52.595172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa9901-d0fb-4026-8399-54065f36fbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:52.596084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa9901-d0fb-4026-8399-54065f36fbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:52.596498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa9901-d0fb-4026-8399-54065f36fbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:52.596931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa9901-d0fb-4026-8399-54065f36fbe5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:52.604592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa9901-d0fb-4026-8399-54065f36fbe5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:52.606252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa9901-d0fb-4026-8399-54065f36fbe5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-fffa9901-d0fb-4026-8399-54065f36fbe5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:52.618949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:52.620135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:52.621325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:52.621325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:52.621325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:52.621325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:52.621325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:52.621325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:52.621325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:52.621325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:52.621325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:52.621325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:52.623766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 491/964] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:52.630202 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ff7d0-2e7f-433b-9b09-40ef9bb59337 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:52.661703 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:52.739233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ff7d0-2e7f-433b-9b09-40ef9bb59337 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:52.774210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdbc4d-5529-46db-881c-9d7581d42175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:52.774816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdbc4d-5529-46db-881c-9d7581d42175 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:52.775092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdbc4d-5529-46db-881c-9d7581d42175 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:52.775358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdbc4d-5529-46db-881c-9d7581d42175 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:52.776217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdbc4d-5529-46db-881c-9d7581d42175 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:52.776521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdbc4d-5529-46db-881c-9d7581d42175 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:52.778209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdbc4d-5529-46db-881c-9d7581d42175 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:52.817192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 474/965] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 12:20:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:52.934010 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a3940-b39b-44cd-8c57-9c76dd9efbf8 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:52.934378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a3940-b39b-44cd-8c57-9c76dd9efbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:52.934559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a3940-b39b-44cd-8c57-9c76dd9efbf8 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:52.936263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 492/966] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:52.942399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb8b3f-1c5b-4c9c-bcd6-f54c94fa91f9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:52.942710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb8b3f-1c5b-4c9c-bcd6-f54c94fa91f9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:52.942977 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb8b3f-1c5b-4c9c-bcd6-f54c94fa91f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:52.970600 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f6eb8b3f-1c5b-4c9c-bcd6-f54c94fa91f9 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:52.972137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f6eb8b3f-1c5b-4c9c-bcd6-f54c94fa91f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:53.049803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 475/967] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:53.094723 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cd0c9-b2a9-4acc-82cf-df18c1637eec None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:53.095061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cd0c9-b2a9-4acc-82cf-df18c1637eec None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:53.095280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cd0c9-b2a9-4acc-82cf-df18c1637eec None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:53.096997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 493/968] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:53.103794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df150864-a5a3-41e4-9360-965150776416 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:53.103794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df150864-a5a3-41e4-9360-965150776416 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:53.103794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df150864-a5a3-41e4-9360-965150776416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:53.121945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-df150864-a5a3-41e4-9360-965150776416 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:53.123144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-df150864-a5a3-41e4-9360-965150776416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:53.134747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:53.194887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 476/969] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:53.330526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cd0c9-b2a9-4acc-82cf-df18c1637eec None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:53.391740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:53.436141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cd0c9-b2a9-4acc-82cf-df18c1637eec None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:53.458518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df150864-a5a3-41e4-9360-965150776416 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:53.471973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa97204-5962-460e-bf5a-b84066727a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:53.472597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa97204-5962-460e-bf5a-b84066727a52 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:53.472916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa97204-5962-460e-bf5a-b84066727a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:53.473174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa97204-5962-460e-bf5a-b84066727a52 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:53.473771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa97204-5962-460e-bf5a-b84066727a52 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:53.475143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa97204-5962-460e-bf5a-b84066727a52 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3fa97204-5962-460e-bf5a-b84066727a52 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:53.484371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:53.485466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:53.486324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:53.488834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 494/970] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:53.491590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa97204-5962-460e-bf5a-b84066727a52 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:53.562079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df150864-a5a3-41e4-9360-965150776416 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:53.593614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa97204-5962-460e-bf5a-b84066727a52 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:53.597681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11711e88-c9ed-425f-b95a-faa6c68afbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:53.598316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11711e88-c9ed-425f-b95a-faa6c68afbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:53.598594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11711e88-c9ed-425f-b95a-faa6c68afbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:53.598861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11711e88-c9ed-425f-b95a-faa6c68afbd0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:53.609583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11711e88-c9ed-425f-b95a-faa6c68afbd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-11711e88-c9ed-425f-b95a-faa6c68afbd0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:53.614235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:53.615330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:53.616280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 477/971] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 24 12:20:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:53.617063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5732-62f3-43ed-ba61-2f02e8bc5cc3 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:53.617063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5732-62f3-43ed-ba61-2f02e8bc5cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:53.617063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5732-62f3-43ed-ba61-2f02e8bc5cc3 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:53.619520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:53.620161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 478/972] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:53.625150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb5732-62f3-43ed-ba61-2f02e8bc5cc3 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:53.632470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858f6f0-8ae5-48d7-bace-98ec53a10dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:53.633109 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858f6f0-8ae5-48d7-bace-98ec53a10dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:53.633385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858f6f0-8ae5-48d7-bace-98ec53a10dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:53.633653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858f6f0-8ae5-48d7-bace-98ec53a10dc4 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:53.634185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:53.634303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858f6f0-8ae5-48d7-bace-98ec53a10dc4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:53.634600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858f6f0-8ae5-48d7-bace-98ec53a10dc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:53.636251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858f6f0-8ae5-48d7-bace-98ec53a10dc4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:53.641437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 495/973] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:53.642626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0e3af-0a54-4e65-becc-d840d853ccc1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:53.642854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0e3af-0a54-4e65-becc-d840d853ccc1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:53.643126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0e3af-0a54-4e65-becc-d840d853ccc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:53.661397 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-eca0e3af-0a54-4e65-becc-d840d853ccc1 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:53.662607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-eca0e3af-0a54-4e65-becc-d840d853ccc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:53.728940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 496/974] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:53.729683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0e3af-0a54-4e65-becc-d840d853ccc1 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:53.741382 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb5732-62f3-43ed-ba61-2f02e8bc5cc3 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:53.777707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e199c5-af39-450e-b417-0c03be840dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:53.778330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e199c5-af39-450e-b417-0c03be840dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:53.778612 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e199c5-af39-450e-b417-0c03be840dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:53.778856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e199c5-af39-450e-b417-0c03be840dd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:53.779516 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e199c5-af39-450e-b417-0c03be840dd1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:53.779815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e199c5-af39-450e-b417-0c03be840dd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:53.781246 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e199c5-af39-450e-b417-0c03be840dd1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:53.789722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 479/975] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 12:20:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:53.796056 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e199c5-af39-450e-b417-0c03be840dd1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:53.837437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0e3af-0a54-4e65-becc-d840d853ccc1 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:53.855310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:53.869090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:53.877309 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a9a5f-7597-4d7b-a485-38e8a26bdb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:53.877920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a9a5f-7597-4d7b-a485-38e8a26bdb3a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:53.878196 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a9a5f-7597-4d7b-a485-38e8a26bdb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:53.878462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a9a5f-7597-4d7b-a485-38e8a26bdb3a None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:53.879097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a9a5f-7597-4d7b-a485-38e8a26bdb3a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:53.879396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a9a5f-7597-4d7b-a485-38e8a26bdb3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:53.880804 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a9a5f-7597-4d7b-a485-38e8a26bdb3a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:53.885983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 497/976] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:53.891734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a9a5f-7597-4d7b-a485-38e8a26bdb3a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:53.905106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e199c5-af39-450e-b417-0c03be840dd1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:53.940896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd32a2b-5668-4154-bc6f-4dbbf434a652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:53.941521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd32a2b-5668-4154-bc6f-4dbbf434a652 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:53.941795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd32a2b-5668-4154-bc6f-4dbbf434a652 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:53.942061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd32a2b-5668-4154-bc6f-4dbbf434a652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:53.953529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd32a2b-5668-4154-bc6f-4dbbf434a652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:53.954991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd32a2b-5668-4154-bc6f-4dbbf434a652 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-abd32a2b-5668-4154-bc6f-4dbbf434a652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:53.970141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:53.971114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:53.971985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:53.971985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:53.971985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:53.971985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 480/977] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:53.980597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd32a2b-5668-4154-bc6f-4dbbf434a652 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:54.007978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a9a5f-7597-4d7b-a485-38e8a26bdb3a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:54.042934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a88c8-e4fa-48b8-af0b-9da8b39d1e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:54.043534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a88c8-e4fa-48b8-af0b-9da8b39d1e46 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.044081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a88c8-e4fa-48b8-af0b-9da8b39d1e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.044081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a88c8-e4fa-48b8-af0b-9da8b39d1e46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.044807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a88c8-e4fa-48b8-af0b-9da8b39d1e46 None admin] RBAC: Adding query filter params (interface=admin, service_id=8acea1b94fa94e6e98025f17920f0b50, region_id=RegionOne) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:54.045115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a88c8-e4fa-48b8-af0b-9da8b39d1e46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:54.046551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a88c8-e4fa-48b8-af0b-9da8b39d1e46 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:54.051608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 498/978] 10.222.0.27 () {56 vars in 1249 bytes} [Wed Aug 24 12:20:53 2022] GET /identity/v3/endpoints?service_id=8acea1b94fa94e6e98025f17920f0b50&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:54.052261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a88c8-e4fa-48b8-af0b-9da8b39d1e46 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:54.099484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd32a2b-5668-4154-bc6f-4dbbf434a652 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:54.105253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:54.119950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:54.135621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d99b6-7ba5-4b12-9f80-fe7a747ff537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:54.136226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d99b6-7ba5-4b12-9f80-fe7a747ff537 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.136505 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d99b6-7ba5-4b12-9f80-fe7a747ff537 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.136814 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d99b6-7ba5-4b12-9f80-fe7a747ff537 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.137468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d99b6-7ba5-4b12-9f80-fe7a747ff537 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:54.137765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d99b6-7ba5-4b12-9f80-fe7a747ff537 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:54.139165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d99b6-7ba5-4b12-9f80-fe7a747ff537 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:54.151262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 481/979] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:54.151954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d99b6-7ba5-4b12-9f80-fe7a747ff537 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:54.164017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a88c8-e4fa-48b8-af0b-9da8b39d1e46 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:54.198825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fa935-db42-4082-9916-90573ba358e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:54.199528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fa935-db42-4082-9916-90573ba358e9 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.199811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fa935-db42-4082-9916-90573ba358e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.200092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fa935-db42-4082-9916-90573ba358e9 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.200735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fa935-db42-4082-9916-90573ba358e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:54.202133 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fa935-db42-4082-9916-90573ba358e9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:54.218649 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 499/980] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:54 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:54.219442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fa935-db42-4082-9916-90573ba358e9 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:54.271687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d99b6-7ba5-4b12-9f80-fe7a747ff537 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:54.321471 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641b082-d9e4-4d42-af36-23d6ddc0e62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:54.322552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641b082-d9e4-4d42-af36-23d6ddc0e62c None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.323035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641b082-d9e4-4d42-af36-23d6ddc0e62c None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.323314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641b082-d9e4-4d42-af36-23d6ddc0e62c None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.323947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641b082-d9e4-4d42-af36-23d6ddc0e62c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:54.325409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641b082-d9e4-4d42-af36-23d6ddc0e62c None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:54.336694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 482/981] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:54 2022] GET /identity/v3/services => generated 1727 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:20:54.349143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fa935-db42-4082-9916-90573ba358e9 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:54.353026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:54.394555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc956f-6f4c-412c-834f-780291a59289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:54.395209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc956f-6f4c-412c-834f-780291a59289 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.395485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc956f-6f4c-412c-834f-780291a59289 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.395753 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc956f-6f4c-412c-834f-780291a59289 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.405577 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc956f-6f4c-412c-834f-780291a59289 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:54.407109 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc956f-6f4c-412c-834f-780291a59289 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-8acc956f-6f4c-412c-834f-780291a59289 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:54.420278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:54.421370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:54.422296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:54.422296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:54.422296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:54.422296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:54.422296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:54.422296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:54.422296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:54.422296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:54.422296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:54.422296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:54.422296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 500/982] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 12:20:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:54.429453 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641b082-d9e4-4d42-af36-23d6ddc0e62c None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:54.478164 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:54.550072 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641b082-d9e4-4d42-af36-23d6ddc0e62c None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:54.585351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211f266-380d-4aec-bd33-15908a654963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:54.585966 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211f266-380d-4aec-bd33-15908a654963 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.586233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211f266-380d-4aec-bd33-15908a654963 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.586499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211f266-380d-4aec-bd33-15908a654963 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.587553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211f266-380d-4aec-bd33-15908a654963 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:54.587845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211f266-380d-4aec-bd33-15908a654963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:54.589757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211f266-380d-4aec-bd33-15908a654963 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:54.609610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 483/983] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 24 12:20:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:54.615178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc956f-6f4c-412c-834f-780291a59289 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:54.719912 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc956f-6f4c-412c-834f-780291a59289 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:54.753970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3bbc4-d685-480d-803a-715cd88a081b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:54.754572 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3bbc4-d685-480d-803a-715cd88a081b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.754843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3bbc4-d685-480d-803a-715cd88a081b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.755104 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3bbc4-d685-480d-803a-715cd88a081b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.786985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:54.827955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3bbc4-d685-480d-803a-715cd88a081b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:54.829495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3bbc4-d685-480d-803a-715cd88a081b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:54.929755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 501/984] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:54 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:20:54.935564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211f266-380d-4aec-bd33-15908a654963 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.042291 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:55.046970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211f266-380d-4aec-bd33-15908a654963 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.087656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9679b3-585d-4cf5-ac17-2b6b774c9f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.088557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9679b3-585d-4cf5-ac17-2b6b774c9f0d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.088970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9679b3-585d-4cf5-ac17-2b6b774c9f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.089151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9679b3-585d-4cf5-ac17-2b6b774c9f0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.099519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9679b3-585d-4cf5-ac17-2b6b774c9f0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.100987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9679b3-585d-4cf5-ac17-2b6b774c9f0d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-1f9679b3-585d-4cf5-ac17-2b6b774c9f0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:55.118233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:55.119319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:55.120627 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:55.120627 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:55.120627 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:55.124682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 484/985] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:55.125460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3bbc4-d685-480d-803a-715cd88a081b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.188589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:55.232514 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3bbc4-d685-480d-803a-715cd88a081b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.268308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af56a90-0733-4c45-b625-2350fb49a687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.268954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af56a90-0733-4c45-b625-2350fb49a687 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.269202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af56a90-0733-4c45-b625-2350fb49a687 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.269497 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af56a90-0733-4c45-b625-2350fb49a687 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.270114 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af56a90-0733-4c45-b625-2350fb49a687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:55.270444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af56a90-0733-4c45-b625-2350fb49a687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.271809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af56a90-0733-4c45-b625-2350fb49a687 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.284053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 502/986] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:55.289631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9679b3-585d-4cf5-ac17-2b6b774c9f0d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.414478 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9679b3-585d-4cf5-ac17-2b6b774c9f0d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.452033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b48620-8d3a-4fc8-8f3e-ad41a8039d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.452675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b48620-8d3a-4fc8-8f3e-ad41a8039d9a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.452964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b48620-8d3a-4fc8-8f3e-ad41a8039d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.453235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b48620-8d3a-4fc8-8f3e-ad41a8039d9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.460105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b48620-8d3a-4fc8-8f3e-ad41a8039d9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.461582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b48620-8d3a-4fc8-8f3e-ad41a8039d9a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-47b48620-8d3a-4fc8-8f3e-ad41a8039d9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:55.473152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:55.474084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:55.474961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:55.474961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:55.474961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:55.474961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:55.474961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:55.474961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:55.474961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:55.474961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:55.474961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 485/987] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:55.482154 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af56a90-0733-4c45-b625-2350fb49a687 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.584118 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af56a90-0733-4c45-b625-2350fb49a687 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.608234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:55.620115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ebd67-c03f-4469-9bc1-5c581a361883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.620751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ebd67-c03f-4469-9bc1-5c581a361883 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.621008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ebd67-c03f-4469-9bc1-5c581a361883 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.621275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ebd67-c03f-4469-9bc1-5c581a361883 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.622026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ebd67-c03f-4469-9bc1-5c581a361883 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:55.622328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ebd67-c03f-4469-9bc1-5c581a361883 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.623756 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ebd67-c03f-4469-9bc1-5c581a361883 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.636243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 503/988] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:55.642377 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b48620-8d3a-4fc8-8f3e-ad41a8039d9a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.657041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:55.746977 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b48620-8d3a-4fc8-8f3e-ad41a8039d9a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.783857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d0ebb-158f-4f0d-988c-d9283915c5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.784460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d0ebb-158f-4f0d-988c-d9283915c5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.784761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d0ebb-158f-4f0d-988c-d9283915c5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.785029 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d0ebb-158f-4f0d-988c-d9283915c5d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.785756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d0ebb-158f-4f0d-988c-d9283915c5d5 None admin] RBAC: Adding query filter params (role.id=1b4412853cfe43d3b524d26be06750ef, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=34a3cadee8ea4b3e9dff0bc23dfe9dc5) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:55.786057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d0ebb-158f-4f0d-988c-d9283915c5d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.787496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d0ebb-158f-4f0d-988c-d9283915c5d5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.793777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 486/989] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:55 2022] GET /identity/v3/role_assignments?user.id=34a3cadee8ea4b3e9dff0bc23dfe9dc5&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=1b4412853cfe43d3b524d26be06750ef => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:56.175343 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fd577-7abe-4ae4-803d-87735559a402 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.175852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fd577-7abe-4ae4-803d-87735559a402 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.176289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fd577-7abe-4ae4-803d-87735559a402 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.178243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 504/990] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:56.184055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581072d-b94e-40f8-b937-34988557257b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.184344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581072d-b94e-40f8-b937-34988557257b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.184620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581072d-b94e-40f8-b937-34988557257b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.203567 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b581072d-b94e-40f8-b937-34988557257b None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:56.204795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b581072d-b94e-40f8-b937-34988557257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:56.261722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b76e1-75e8-4235-b105-81e275c04843 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.261909 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b76e1-75e8-4235-b105-81e275c04843 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.262166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b76e1-75e8-4235-b105-81e275c04843 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.263672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 505/991] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:56.268636 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dea0bb-dca3-4e67-93b6-8755a967cd1a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.268958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dea0bb-dca3-4e67-93b6-8755a967cd1a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.269237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dea0bb-dca3-4e67-93b6-8755a967cd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.275105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 487/992] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:56.287722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f9dea0bb-dca3-4e67-93b6-8755a967cd1a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:56.288935 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f9dea0bb-dca3-4e67-93b6-8755a967cd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:56.359010 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 506/993] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:56.539952 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581072d-b94e-40f8-b937-34988557257b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.543690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:56.622510 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dea0bb-dca3-4e67-93b6-8755a967cd1a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.654436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581072d-b94e-40f8-b937-34988557257b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:56.691088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09018a-e6e7-41ba-9b07-2600b2998a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:56.691717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09018a-e6e7-41ba-9b07-2600b2998a8d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.691990 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09018a-e6e7-41ba-9b07-2600b2998a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.692261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09018a-e6e7-41ba-9b07-2600b2998a8d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.692861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09018a-e6e7-41ba-9b07-2600b2998a8d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:56.694241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09018a-e6e7-41ba-9b07-2600b2998a8d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-db09018a-e6e7-41ba-9b07-2600b2998a8d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:56.705984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:56.707199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:56.708059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 488/994] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:56.713201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09018a-e6e7-41ba-9b07-2600b2998a8d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.732666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dea0bb-dca3-4e67-93b6-8755a967cd1a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:56.747820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:56.770847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3f974-ce88-4577-a777-bbe0a4d1546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:56.771480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3f974-ce88-4577-a777-bbe0a4d1546d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.771761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3f974-ce88-4577-a777-bbe0a4d1546d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.772023 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3f974-ce88-4577-a777-bbe0a4d1546d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.772619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3f974-ce88-4577-a777-bbe0a4d1546d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:56.774111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3f974-ce88-4577-a777-bbe0a4d1546d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-03a3f974-ce88-4577-a777-bbe0a4d1546d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:56.783026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:56.784035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:56.784930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 507/995] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:56.787204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:56.790202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3f974-ce88-4577-a777-bbe0a4d1546d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.823622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09018a-e6e7-41ba-9b07-2600b2998a8d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:56.859352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd61ed-ddd1-4573-816a-15f09af5d9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:56.859961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd61ed-ddd1-4573-816a-15f09af5d9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.860240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd61ed-ddd1-4573-816a-15f09af5d9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.860503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd61ed-ddd1-4573-816a-15f09af5d9c6 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.861169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd61ed-ddd1-4573-816a-15f09af5d9c6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:56.861453 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd61ed-ddd1-4573-816a-15f09af5d9c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:56.862901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd61ed-ddd1-4573-816a-15f09af5d9c6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:56.867782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 489/996] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:56.873616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd61ed-ddd1-4573-816a-15f09af5d9c6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.904896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3f974-ce88-4577-a777-bbe0a4d1546d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:56.943518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3203d12-8f01-4474-831d-19cd61c747df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:56.944714 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3203d12-8f01-4474-831d-19cd61c747df None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.944989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3203d12-8f01-4474-831d-19cd61c747df None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.945300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3203d12-8f01-4474-831d-19cd61c747df None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.945932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3203d12-8f01-4474-831d-19cd61c747df None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:56.946231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3203d12-8f01-4474-831d-19cd61c747df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:56.947634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3203d12-8f01-4474-831d-19cd61c747df None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:56.956679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 508/997] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:56.958606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3203d12-8f01-4474-831d-19cd61c747df None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.985478 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd61ed-ddd1-4573-816a-15f09af5d9c6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:57.009880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:57.018500 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:57.023848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05497cb-3ea0-4dc5-9b59-e622e9f20243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:57.024455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05497cb-3ea0-4dc5-9b59-e622e9f20243 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:57.024762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05497cb-3ea0-4dc5-9b59-e622e9f20243 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:57.025049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05497cb-3ea0-4dc5-9b59-e622e9f20243 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:57.025730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05497cb-3ea0-4dc5-9b59-e622e9f20243 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:57.026103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05497cb-3ea0-4dc5-9b59-e622e9f20243 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:57.027647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05497cb-3ea0-4dc5-9b59-e622e9f20243 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:57.033421 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 490/998] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:57.039276 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05497cb-3ea0-4dc5-9b59-e622e9f20243 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:57.072881 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3203d12-8f01-4474-831d-19cd61c747df None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:57.118754 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2de35e-8bfa-402d-9881-7487a2287ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:57.120699 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2de35e-8bfa-402d-9881-7487a2287ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:57.121187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2de35e-8bfa-402d-9881-7487a2287ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:57.121679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2de35e-8bfa-402d-9881-7487a2287ff2 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:57.122513 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2de35e-8bfa-402d-9881-7487a2287ff2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:57.123025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2de35e-8bfa-402d-9881-7487a2287ff2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:57.125144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2de35e-8bfa-402d-9881-7487a2287ff2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:57.132825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 509/999] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:57.138038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2de35e-8bfa-402d-9881-7487a2287ff2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:57.151166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05497cb-3ea0-4dc5-9b59-e622e9f20243 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:57.187090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15d9e5-575c-4b30-bcd8-6a6433cebc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:57.187596 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15d9e5-575c-4b30-bcd8-6a6433cebc39 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:57.187874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15d9e5-575c-4b30-bcd8-6a6433cebc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:57.188142 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15d9e5-575c-4b30-bcd8-6a6433cebc39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:57.188879 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15d9e5-575c-4b30-bcd8-6a6433cebc39 None admin] RBAC: Adding query filter params (interface=public, service_id=1651ad5ab2334cb2b743b81b48274397, region_id=RegionOne) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:57.189184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15d9e5-575c-4b30-bcd8-6a6433cebc39 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:57.190774 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15d9e5-575c-4b30-bcd8-6a6433cebc39 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:57.195965 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 491/1000] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:57 2022] GET /identity/v3/endpoints?service_id=1651ad5ab2334cb2b743b81b48274397&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:57.201454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15d9e5-575c-4b30-bcd8-6a6433cebc39 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:57.244710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2de35e-8bfa-402d-9881-7487a2287ff2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:57.251959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:57.269670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:57.281104 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd2466-48d1-4f75-adca-703f9c49a1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:57.281712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd2466-48d1-4f75-adca-703f9c49a1a3 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:57.281986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd2466-48d1-4f75-adca-703f9c49a1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:57.282360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd2466-48d1-4f75-adca-703f9c49a1a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:57.283650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd2466-48d1-4f75-adca-703f9c49a1a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:57.284495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd2466-48d1-4f75-adca-703f9c49a1a3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:57.307754 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 510/1001] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:57.325151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15d9e5-575c-4b30-bcd8-6a6433cebc39 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:57.373290 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c2b97-be80-4c46-afc7-181930cae615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:57.373906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c2b97-be80-4c46-afc7-181930cae615 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:57.374184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c2b97-be80-4c46-afc7-181930cae615 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:57.374452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c2b97-be80-4c46-afc7-181930cae615 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:57.375061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c2b97-be80-4c46-afc7-181930cae615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:57.376493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c2b97-be80-4c46-afc7-181930cae615 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:57.392715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 492/1002] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:57 2022] GET /identity/v3/services => generated 1727 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:20:57.773517 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5181a64-2c00-4707-85b9-a17937fbd7aa None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:57.774094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5181a64-2c00-4707-85b9-a17937fbd7aa None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:57.774094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5181a64-2c00-4707-85b9-a17937fbd7aa None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:57.775590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 511/1003] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:57.780969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e96e8-08c1-4720-aae3-9c4c230b1891 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:57.781433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e96e8-08c1-4720-aae3-9c4c230b1891 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:57.781509 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e96e8-08c1-4720-aae3-9c4c230b1891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:57.800122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-086e96e8-08c1-4720-aae3-9c4c230b1891 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:57.801340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-086e96e8-08c1-4720-aae3-9c4c230b1891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:57.872118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 493/1004] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:57 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:58.135646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5181a64-2c00-4707-85b9-a17937fbd7aa None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:58.252041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5181a64-2c00-4707-85b9-a17937fbd7aa None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:58.287620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5637e21-591e-4daf-b945-8607ee842614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:58.288253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5637e21-591e-4daf-b945-8607ee842614 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:58.288528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5637e21-591e-4daf-b945-8607ee842614 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:58.288842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5637e21-591e-4daf-b945-8607ee842614 None admin] PATH_INFO: `/v3/services/image` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:58.289423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5637e21-591e-4daf-b945-8607ee842614 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:58.290795 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5637e21-591e-4daf-b945-8607ee842614 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-b5637e21-591e-4daf-b945-8607ee842614 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:58.299747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:58.300831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:20:58.302281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 512/1005] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Aug 24 12:20:58 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:58.309976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e96e8-08c1-4720-aae3-9c4c230b1891 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:58.367312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:58.423996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e96e8-08c1-4720-aae3-9c4c230b1891 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:58.462524 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937212e-85d5-4246-a113-7b3bca250308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:58.463214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937212e-85d5-4246-a113-7b3bca250308 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:58.463479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937212e-85d5-4246-a113-7b3bca250308 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:58.463744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937212e-85d5-4246-a113-7b3bca250308 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:58.464422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937212e-85d5-4246-a113-7b3bca250308 None admin] RBAC: Adding query filter params (name=image) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:58.464754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937212e-85d5-4246-a113-7b3bca250308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:58.466185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937212e-85d5-4246-a113-7b3bca250308 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:58.472152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 494/1006] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:58.477799 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5637e21-591e-4daf-b945-8607ee842614 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:58.509924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:58.584203 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5637e21-591e-4daf-b945-8607ee842614 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:58.618330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a8ff8-4d98-4316-9039-5ec90efcd0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:58.619327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a8ff8-4d98-4316-9039-5ec90efcd0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:58.619770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a8ff8-4d98-4316-9039-5ec90efcd0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:58.620195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a8ff8-4d98-4316-9039-5ec90efcd0e5 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:58.621006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a8ff8-4d98-4316-9039-5ec90efcd0e5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:58.621454 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a8ff8-4d98-4316-9039-5ec90efcd0e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:58.622991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a8ff8-4d98-4316-9039-5ec90efcd0e5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:58.628056 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 513/1007] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:58 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:59.235125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a51bd-96df-4bb8-af0f-1fd953a9d1db None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.235645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a51bd-96df-4bb8-af0f-1fd953a9d1db None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.235645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a51bd-96df-4bb8-af0f-1fd953a9d1db None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.236924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 495/1008] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:59.242743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76fc24-da51-427c-8301-af3d784bc9fe None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.243166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76fc24-da51-427c-8301-af3d784bc9fe None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.243267 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76fc24-da51-427c-8301-af3d784bc9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.263213 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8b76fc24-da51-427c-8301-af3d784bc9fe None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:59.264382 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8b76fc24-da51-427c-8301-af3d784bc9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:59.290489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41337e84-8bf6-4228-bebe-649fb42e7c4f None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.290787 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41337e84-8bf6-4228-bebe-649fb42e7c4f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.291628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41337e84-8bf6-4228-bebe-649fb42e7c4f None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.293760 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 496/1009] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:59.298499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc417fc-10d2-4aff-99c3-35e47f15e34d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.298647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc417fc-10d2-4aff-99c3-35e47f15e34d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.298955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc417fc-10d2-4aff-99c3-35e47f15e34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.317959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3dc417fc-10d2-4aff-99c3-35e47f15e34d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:59.319106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3dc417fc-10d2-4aff-99c3-35e47f15e34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:59.350369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 514/1010] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:59 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:59.393941 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 497/1011] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:59 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:59.615576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76fc24-da51-427c-8301-af3d784bc9fe None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:59.646776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:59.658823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc417fc-10d2-4aff-99c3-35e47f15e34d None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:59.707093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:59.734030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76fc24-da51-427c-8301-af3d784bc9fe None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.767382 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc417fc-10d2-4aff-99c3-35e47f15e34d None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.778449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e995ee-3ae0-4f3b-a470-efcf8d15a065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.779477 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e995ee-3ae0-4f3b-a470-efcf8d15a065 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.779653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e995ee-3ae0-4f3b-a470-efcf8d15a065 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.779920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e995ee-3ae0-4f3b-a470-efcf8d15a065 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.781173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e995ee-3ae0-4f3b-a470-efcf8d15a065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:59.782635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e995ee-3ae0-4f3b-a470-efcf8d15a065 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:59.784557 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 515/1012] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:59.790011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e995ee-3ae0-4f3b-a470-efcf8d15a065 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:59.808996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516eef0-11fe-402c-b7ed-1d2694f82aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.809724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516eef0-11fe-402c-b7ed-1d2694f82aaf None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.809996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516eef0-11fe-402c-b7ed-1d2694f82aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.810264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516eef0-11fe-402c-b7ed-1d2694f82aaf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.810847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516eef0-11fe-402c-b7ed-1d2694f82aaf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:59.812225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516eef0-11fe-402c-b7ed-1d2694f82aaf None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b516eef0-11fe-402c-b7ed-1d2694f82aaf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:59.820980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:59.821943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:20:59.822961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 498/1013] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:59.828094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516eef0-11fe-402c-b7ed-1d2694f82aaf None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:59.892762 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:59.905628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e995ee-3ae0-4f3b-a470-efcf8d15a065 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.939442 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516eef0-11fe-402c-b7ed-1d2694f82aaf None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.941007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdaa9de-5137-405a-b430-dcf14357b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.941616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdaa9de-5137-405a-b430-dcf14357b2f2 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.941891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdaa9de-5137-405a-b430-dcf14357b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.942159 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdaa9de-5137-405a-b430-dcf14357b2f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.942797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdaa9de-5137-405a-b430-dcf14357b2f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:59.944230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdaa9de-5137-405a-b430-dcf14357b2f2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:59.950939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:59.972700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 516/1014] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:59 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:20:59.993036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59877412-7e5f-4c10-87c5-7ca9a6b22ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.993763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59877412-7e5f-4c10-87c5-7ca9a6b22ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.994151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59877412-7e5f-4c10-87c5-7ca9a6b22ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.994509 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59877412-7e5f-4c10-87c5-7ca9a6b22ce1 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.995234 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59877412-7e5f-4c10-87c5-7ca9a6b22ce1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:59.995618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59877412-7e5f-4c10-87c5-7ca9a6b22ce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:59.997316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59877412-7e5f-4c10-87c5-7ca9a6b22ce1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:00.003769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 499/1015] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:00.010114 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdaa9de-5137-405a-b430-dcf14357b2f2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:00.122829 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdaa9de-5137-405a-b430-dcf14357b2f2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:00.158654 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60496031-2ed3-4122-a5ca-4453681c72ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:00.159168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60496031-2ed3-4122-a5ca-4453681c72ee None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:00.159444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60496031-2ed3-4122-a5ca-4453681c72ee None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:00.159712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60496031-2ed3-4122-a5ca-4453681c72ee None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:00.160353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60496031-2ed3-4122-a5ca-4453681c72ee None admin] RBAC: Adding query filter params (type=compute) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:00.160631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60496031-2ed3-4122-a5ca-4453681c72ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:00.162024 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60496031-2ed3-4122-a5ca-4453681c72ee None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:00.167389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 517/1016] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 12:21:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:00.173368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59877412-7e5f-4c10-87c5-7ca9a6b22ce1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:00.285837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59877412-7e5f-4c10-87c5-7ca9a6b22ce1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:00.322803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e0af3-1d7e-4554-88e3-a6d53c031dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:00.323806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e0af3-1d7e-4554-88e3-a6d53c031dfb None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:00.324062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e0af3-1d7e-4554-88e3-a6d53c031dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:00.324350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e0af3-1d7e-4554-88e3-a6d53c031dfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:00.325052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e0af3-1d7e-4554-88e3-a6d53c031dfb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:00.326500 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e0af3-1d7e-4554-88e3-a6d53c031dfb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:00.351062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 500/1017] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:21:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:00.592555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bca8b1-4f7a-4345-905d-461d8695a6f6 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:00.593094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bca8b1-4f7a-4345-905d-461d8695a6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:00.593094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bca8b1-4f7a-4345-905d-461d8695a6f6 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:00.594551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 518/1018] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:00.600043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef0304-6e94-4e32-96ab-d8a205c0281a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:00.600559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef0304-6e94-4e32-96ab-d8a205c0281a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:00.600938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef0304-6e94-4e32-96ab-d8a205c0281a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:00.623321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-edef0304-6e94-4e32-96ab-d8a205c0281a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:00.624537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-edef0304-6e94-4e32-96ab-d8a205c0281a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:00.649843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:00.707858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 501/1019] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:00 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:00.971298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bca8b1-4f7a-4345-905d-461d8695a6f6 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:00.984517 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:01.081940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bca8b1-4f7a-4345-905d-461d8695a6f6 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:01.120393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6100dc1-0149-466d-9f37-c03620da8b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:01.121034 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6100dc1-0149-466d-9f37-c03620da8b99 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:01.121312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6100dc1-0149-466d-9f37-c03620da8b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:01.121583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6100dc1-0149-466d-9f37-c03620da8b99 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:01.122233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6100dc1-0149-466d-9f37-c03620da8b99 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:01.123614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6100dc1-0149-466d-9f37-c03620da8b99 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:01.143011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 519/1020] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:21:00 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:01.915349 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744230b6-493f-480f-ad9f-aaba614ed7bf None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:01.915911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744230b6-493f-480f-ad9f-aaba614ed7bf None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:01.915911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744230b6-493f-480f-ad9f-aaba614ed7bf None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:01.917419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 502/1021] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:01.922634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef5bb6-c8d7-4b68-9bf3-1c4f104d5a8d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:01.923042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef5bb6-c8d7-4b68-9bf3-1c4f104d5a8d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:01.923200 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef5bb6-c8d7-4b68-9bf3-1c4f104d5a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:01.941834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e8ef5bb6-c8d7-4b68-9bf3-1c4f104d5a8d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:01.943027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e8ef5bb6-c8d7-4b68-9bf3-1c4f104d5a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:02.013409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:02.032693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 520/1022] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:02.290237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744230b6-493f-480f-ad9f-aaba614ed7bf None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:02.361312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b10c2f-dca4-4b02-981c-fb23f5ee50e5 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:02.361590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b10c2f-dca4-4b02-981c-fb23f5ee50e5 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:02.361864 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b10c2f-dca4-4b02-981c-fb23f5ee50e5 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:02.363224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 521/1023] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:21:02.368127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a588e1-ef4d-466c-9d59-6efd120374c9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:02.368408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a588e1-ef4d-466c-9d59-6efd120374c9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:02.368704 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a588e1-ef4d-466c-9d59-6efd120374c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:02.386802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b0a588e1-ef4d-466c-9d59-6efd120374c9 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:02.387978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b0a588e1-ef4d-466c-9d59-6efd120374c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:02.406020 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744230b6-493f-480f-ad9f-aaba614ed7bf None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:02.445548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676de03f-2848-4b96-b0ce-61fd181d53a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:02.446263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676de03f-2848-4b96-b0ce-61fd181d53a8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:02.446441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676de03f-2848-4b96-b0ce-61fd181d53a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:02.446712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676de03f-2848-4b96-b0ce-61fd181d53a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:02.447961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676de03f-2848-4b96-b0ce-61fd181d53a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:02.449375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676de03f-2848-4b96-b0ce-61fd181d53a8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:02.451145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 503/1024] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:02.456880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676de03f-2848-4b96-b0ce-61fd181d53a8 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:02.467430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 522/1025] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:21:02.482826 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:02.562553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676de03f-2848-4b96-b0ce-61fd181d53a8 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:02.599549 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411160f-1741-440c-8335-cb5253970a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:02.600152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411160f-1741-440c-8335-cb5253970a99 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:02.600425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411160f-1741-440c-8335-cb5253970a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:02.600721 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411160f-1741-440c-8335-cb5253970a99 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:02.601362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411160f-1741-440c-8335-cb5253970a99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:02.602808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411160f-1741-440c-8335-cb5253970a99 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:02.632668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 504/1026] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:21:02 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:21:02.737110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a588e1-ef4d-466c-9d59-6efd120374c9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:02.842606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a588e1-ef4d-466c-9d59-6efd120374c9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:02.867625 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:02.880123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fe3b4-8dd3-4b90-a0c0-3a2518ac720e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:02.880798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fe3b4-8dd3-4b90-a0c0-3a2518ac720e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:02.881074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fe3b4-8dd3-4b90-a0c0-3a2518ac720e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:02.881340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fe3b4-8dd3-4b90-a0c0-3a2518ac720e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:02.892069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594fe3b4-8dd3-4b90-a0c0-3a2518ac720e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-594fe3b4-8dd3-4b90-a0c0-3a2518ac720e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:02.896692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:02.897601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:02.898435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 523/1027] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 24 12:21:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:02.904308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411160f-1741-440c-8335-cb5253970a99 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:02.996011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:03.021551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411160f-1741-440c-8335-cb5253970a99 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:03.056752 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218fa75-c367-4243-a02a-153e464d804e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:03.057309 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218fa75-c367-4243-a02a-153e464d804e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.057583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218fa75-c367-4243-a02a-153e464d804e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.057846 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218fa75-c367-4243-a02a-153e464d804e None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.058507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218fa75-c367-4243-a02a-153e464d804e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:03.058801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218fa75-c367-4243-a02a-153e464d804e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:03.060188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218fa75-c367-4243-a02a-153e464d804e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:03.064440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df0a69-adf7-4eb7-a582-be0e9319f47c None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.064940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df0a69-adf7-4eb7-a582-be0e9319f47c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.065017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df0a69-adf7-4eb7-a582-be0e9319f47c None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.066526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 524/1028] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:03.071191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 505/1029] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 12:21:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:03.071396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a0f14-9750-4caa-a040-f211ddbd091a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.071647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a0f14-9750-4caa-a040-f211ddbd091a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.071926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a0f14-9750-4caa-a040-f211ddbd091a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.076744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218fa75-c367-4243-a02a-153e464d804e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:03.090801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3f3a0f14-9750-4caa-a040-f211ddbd091a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:03.091988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3f3a0f14-9750-4caa-a040-f211ddbd091a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:03.166159 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 525/1030] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:03.187451 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218fa75-c367-4243-a02a-153e464d804e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:03.224152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a221773-81fc-4bd2-8b3a-a0744dfc13d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:03.224827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221773-81fc-4bd2-8b3a-a0744dfc13d7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.225108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221773-81fc-4bd2-8b3a-a0744dfc13d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.225380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221773-81fc-4bd2-8b3a-a0744dfc13d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.234808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:03.237104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a221773-81fc-4bd2-8b3a-a0744dfc13d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:03.238624 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a221773-81fc-4bd2-8b3a-a0744dfc13d7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-2a221773-81fc-4bd2-8b3a-a0744dfc13d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:03.254035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:03.255010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:21:03.261375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:03.261375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:03.261375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:03.261375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 506/1031] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:21:03.261613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a0f14-9750-4caa-a040-f211ddbd091a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:03.366412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a0f14-9750-4caa-a040-f211ddbd091a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:03.381237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:03.403069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88553f-76a9-4e66-ab41-2bd706d279c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:03.403684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88553f-76a9-4e66-ab41-2bd706d279c1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.403962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88553f-76a9-4e66-ab41-2bd706d279c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.404229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88553f-76a9-4e66-ab41-2bd706d279c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.404909 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88553f-76a9-4e66-ab41-2bd706d279c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:03.405211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88553f-76a9-4e66-ab41-2bd706d279c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:03.406600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88553f-76a9-4e66-ab41-2bd706d279c1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:03.418381 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 526/1032] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:03.424050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a221773-81fc-4bd2-8b3a-a0744dfc13d7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:03.434414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88553f-76a9-4e66-ab41-2bd706d279c1 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:03.537823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a221773-81fc-4bd2-8b3a-a0744dfc13d7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:03.545781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88553f-76a9-4e66-ab41-2bd706d279c1 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:03.574620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b2af0-f0f1-4a52-a567-70a0ef7daef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:03.575248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b2af0-f0f1-4a52-a567-70a0ef7daef0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.575520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b2af0-f0f1-4a52-a567-70a0ef7daef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.575784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b2af0-f0f1-4a52-a567-70a0ef7daef0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.582038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6df4a-b811-4dc1-bdfe-b2818a879c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:03.582674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6df4a-b811-4dc1-bdfe-b2818a879c4b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.582955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6df4a-b811-4dc1-bdfe-b2818a879c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.583225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6df4a-b811-4dc1-bdfe-b2818a879c4b None admin] PATH_INFO: `/v3/services/image` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.583543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b2af0-f0f1-4a52-a567-70a0ef7daef0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:03.583800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6df4a-b811-4dc1-bdfe-b2818a879c4b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:03.585018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b2af0-f0f1-4a52-a567-70a0ef7daef0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:03.585206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6df4a-b811-4dc1-bdfe-b2818a879c4b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-86c6df4a-b811-4dc1-bdfe-b2818a879c4b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:03.594975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:03.595967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:03.600949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 527/1033] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-826b2af0-f0f1-4a52-a567-70a0ef7daef0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:21:03.601813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:03.602654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:03.604053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:03.604053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:03.604053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:21:03.604053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:21:03.604053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:21:03.604053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:21:03.604053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:21:03.604053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:21:03.604053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:21:03.604053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:03.604053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 507/1034] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:03.605108 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6df4a-b811-4dc1-bdfe-b2818a879c4b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:03.605225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b2af0-f0f1-4a52-a567-70a0ef7daef0 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:03.646918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:03.665774 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:03.716618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6df4a-b811-4dc1-bdfe-b2818a879c4b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:03.717153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b2af0-f0f1-4a52-a567-70a0ef7daef0 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:03.752918 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a7fb9-3c20-4ed1-81b6-af93b6e90009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:03.753525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a7fb9-3c20-4ed1-81b6-af93b6e90009 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.753801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43a0ac-881a-4527-84df-5a7e3742ce14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:03.753936 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a7fb9-3c20-4ed1-81b6-af93b6e90009 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.754099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a7fb9-3c20-4ed1-81b6-af93b6e90009 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.754412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43a0ac-881a-4527-84df-5a7e3742ce14 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.754690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43a0ac-881a-4527-84df-5a7e3742ce14 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.754800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a7fb9-3c20-4ed1-81b6-af93b6e90009 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:03.754969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43a0ac-881a-4527-84df-5a7e3742ce14 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.755062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a7fb9-3c20-4ed1-81b6-af93b6e90009 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:03.755582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43a0ac-881a-4527-84df-5a7e3742ce14 None admin] RBAC: Adding query filter params (name=image) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:03.755943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43a0ac-881a-4527-84df-5a7e3742ce14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:03.756500 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a7fb9-3c20-4ed1-81b6-af93b6e90009 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:03.757351 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43a0ac-881a-4527-84df-5a7e3742ce14 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:03.768302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 528/1035] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:03.769115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43a0ac-881a-4527-84df-5a7e3742ce14 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:03.776298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 508/1036] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:03.782344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a7fb9-3c20-4ed1-81b6-af93b6e90009 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:03.877840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43a0ac-881a-4527-84df-5a7e3742ce14 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:03.879627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:03.885393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a7fb9-3c20-4ed1-81b6-af93b6e90009 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:03.896268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:03.917167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da0cd7-2914-4782-9952-a095c86b0b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:03.917773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da0cd7-2914-4782-9952-a095c86b0b85 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.918168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da0cd7-2914-4782-9952-a095c86b0b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.918286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da0cd7-2914-4782-9952-a095c86b0b85 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.918912 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da0cd7-2914-4782-9952-a095c86b0b85 None admin] RBAC: Adding query filter params (type=image) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:03.919211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da0cd7-2914-4782-9952-a095c86b0b85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:03.920788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da0cd7-2914-4782-9952-a095c86b0b85 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:03.923399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25344c4c-4ff4-46b9-b7cb-0a1863a7a5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:03.924010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25344c4c-4ff4-46b9-b7cb-0a1863a7a5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.924295 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25344c4c-4ff4-46b9-b7cb-0a1863a7a5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.924559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25344c4c-4ff4-46b9-b7cb-0a1863a7a5e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.931791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 529/1037] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:03.932671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da0cd7-2914-4782-9952-a095c86b0b85 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.007162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25344c4c-4ff4-46b9-b7cb-0a1863a7a5e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:04.008638 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25344c4c-4ff4-46b9-b7cb-0a1863a7a5e3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:04.040374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da0cd7-2914-4782-9952-a095c86b0b85 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:04.078864 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98267809-d82e-4f18-906f-e0a03f3842aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:04.079523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98267809-d82e-4f18-906f-e0a03f3842aa None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.079741 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98267809-d82e-4f18-906f-e0a03f3842aa None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.080018 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98267809-d82e-4f18-906f-e0a03f3842aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.080735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98267809-d82e-4f18-906f-e0a03f3842aa None admin] RBAC: Adding query filter params (interface=public, service_id=a598fdb46eb942ffa8683e8987e0b20d, region_id=RegionOne) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:04.081357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98267809-d82e-4f18-906f-e0a03f3842aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:04.082466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98267809-d82e-4f18-906f-e0a03f3842aa None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:04.093806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 530/1038] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/endpoints?service_id=a598fdb46eb942ffa8683e8987e0b20d&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:04.094681 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98267809-d82e-4f18-906f-e0a03f3842aa None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.121111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 509/1039] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 12:21:04.126648 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25344c4c-4ff4-46b9-b7cb-0a1863a7a5e3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.128476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:04.138909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:04.208007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98267809-d82e-4f18-906f-e0a03f3842aa None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:04.231865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25344c4c-4ff4-46b9-b7cb-0a1863a7a5e3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:04.247317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d9446-08b1-4335-8949-befbe82c3ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:04.248041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d9446-08b1-4335-8949-befbe82c3ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.248320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d9446-08b1-4335-8949-befbe82c3ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.248586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d9446-08b1-4335-8949-befbe82c3ff2 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.249347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d9446-08b1-4335-8949-befbe82c3ff2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:04.250730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d9446-08b1-4335-8949-befbe82c3ff2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:04.258182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 531/1040] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 12:21:04 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:21:04.282494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037c4f9-c698-4b91-90fe-49ea6fee8abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:04.283541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037c4f9-c698-4b91-90fe-49ea6fee8abc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.283988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037c4f9-c698-4b91-90fe-49ea6fee8abc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.284456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037c4f9-c698-4b91-90fe-49ea6fee8abc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.297212 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037c4f9-c698-4b91-90fe-49ea6fee8abc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:04.298986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037c4f9-c698-4b91-90fe-49ea6fee8abc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3037c4f9-c698-4b91-90fe-49ea6fee8abc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:04.316260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:04.317401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:21:04.318424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:04.318424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:04.318424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:04.320228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 510/1041] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:21:04.326136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d9446-08b1-4335-8949-befbe82c3ff2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.431669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:04.439740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d9446-08b1-4335-8949-befbe82c3ff2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:04.478607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90899734-8783-47a0-b038-e2436051eb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:04.479568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90899734-8783-47a0-b038-e2436051eb3d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.479926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90899734-8783-47a0-b038-e2436051eb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.480165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90899734-8783-47a0-b038-e2436051eb3d None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.480870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90899734-8783-47a0-b038-e2436051eb3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:04.481173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90899734-8783-47a0-b038-e2436051eb3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:04.482597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90899734-8783-47a0-b038-e2436051eb3d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:04.495362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 532/1042] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:04.500762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037c4f9-c698-4b91-90fe-49ea6fee8abc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.566644 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:04.608536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12645d3-3001-4d0b-ac0a-fc53858e61a9 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.608925 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12645d3-3001-4d0b-ac0a-fc53858e61a9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.609097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12645d3-3001-4d0b-ac0a-fc53858e61a9 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.609582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037c4f9-c698-4b91-90fe-49ea6fee8abc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:04.610633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 533/1043] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:21:04.615475 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fc5a1-3bb4-4d31-b1bc-d02c3a8e48ee None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.615748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fc5a1-3bb4-4d31-b1bc-d02c3a8e48ee None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.616015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fc5a1-3bb4-4d31-b1bc-d02c3a8e48ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.635371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9d8fc5a1-3bb4-4d31-b1bc-d02c3a8e48ee None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:04.636565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9d8fc5a1-3bb4-4d31-b1bc-d02c3a8e48ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:04.645356 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9739b5f-b2f5-4bb2-85d4-4832bb30a882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:04.645988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9739b5f-b2f5-4bb2-85d4-4832bb30a882 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.646350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9739b5f-b2f5-4bb2-85d4-4832bb30a882 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.646615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9739b5f-b2f5-4bb2-85d4-4832bb30a882 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.654070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9739b5f-b2f5-4bb2-85d4-4832bb30a882 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:04.655737 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9739b5f-b2f5-4bb2-85d4-4832bb30a882 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-a9739b5f-b2f5-4bb2-85d4-4832bb30a882 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:21:04.667948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:04.668877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:04.670000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:21:04.670000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:21:04.670000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:21:04.670000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:21:04.670000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:21:04.670000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:21:04.670000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:21:04.670000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:04.670000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 511/1044] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:21:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:04.675674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9739b5f-b2f5-4bb2-85d4-4832bb30a882 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.710419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 534/1045] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:21:04.785905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9739b5f-b2f5-4bb2-85d4-4832bb30a882 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:04.813400 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:04.827158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79ea85-0340-433a-a1a1-5f60444c314e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:04.827768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79ea85-0340-433a-a1a1-5f60444c314e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.828046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79ea85-0340-433a-a1a1-5f60444c314e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.828370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79ea85-0340-433a-a1a1-5f60444c314e None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.829099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79ea85-0340-433a-a1a1-5f60444c314e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:04.829406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79ea85-0340-433a-a1a1-5f60444c314e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:04.830845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79ea85-0340-433a-a1a1-5f60444c314e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:04.844001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 512/1046] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:21:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:04.850282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fc5a1-3bb4-4d31-b1bc-d02c3a8e48ee None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.936127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:04.953972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fc5a1-3bb4-4d31-b1bc-d02c3a8e48ee None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:04.974153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79ea85-0340-433a-a1a1-5f60444c314e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.996513 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48104b8-a0d4-4d89-a6b5-3278109a81c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:04.997257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48104b8-a0d4-4d89-a6b5-3278109a81c4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.997553 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48104b8-a0d4-4d89-a6b5-3278109a81c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.997824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48104b8-a0d4-4d89-a6b5-3278109a81c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.998552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48104b8-a0d4-4d89-a6b5-3278109a81c4 None admin] RBAC: Adding query filter params (role.id=1b4412853cfe43d3b524d26be06750ef, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=e65026e01df242e7be11ed24214a4efa) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:04.998856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48104b8-a0d4-4d89-a6b5-3278109a81c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:05.000390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48104b8-a0d4-4d89-a6b5-3278109a81c4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:05.006791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 535/1047] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:04 2022] GET /identity/v3/role_assignments?user.id=e65026e01df242e7be11ed24214a4efa&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=1b4412853cfe43d3b524d26be06750ef => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:05.083947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79ea85-0340-433a-a1a1-5f60444c314e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:05.120770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfa2c4-0494-477a-9ddd-d1df490532ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:05.121415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfa2c4-0494-477a-9ddd-d1df490532ed None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:05.121689 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfa2c4-0494-477a-9ddd-d1df490532ed None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:05.121952 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfa2c4-0494-477a-9ddd-d1df490532ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:05.133582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfa2c4-0494-477a-9ddd-d1df490532ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-24dfa2c4-0494-477a-9ddd-d1df490532ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:05.138514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:05.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:05.140327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:05.140327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:05.140327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:05.140327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:05.140327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:05.140327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:05.140327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:05.140327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:05.140327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:05.140327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:05.140327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:05.140327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:05.144880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 513/1048] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:05.146014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48104b8-a0d4-4d89-a6b5-3278109a81c4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:05.266065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48104b8-a0d4-4d89-a6b5-3278109a81c4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:05.300824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6386f-2972-4337-8860-d32701238d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:05.301437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6386f-2972-4337-8860-d32701238d00 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:05.301712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6386f-2972-4337-8860-d32701238d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:05.301980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6386f-2972-4337-8860-d32701238d00 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:05.302641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6386f-2972-4337-8860-d32701238d00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:05.302940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6386f-2972-4337-8860-d32701238d00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:05.304317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6386f-2972-4337-8860-d32701238d00 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:05.313744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 536/1049] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:05.319452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfa2c4-0494-477a-9ddd-d1df490532ed None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:05.370572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:05.438827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfa2c4-0494-477a-9ddd-d1df490532ed None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:05.475883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4880b-58c4-4699-b004-743908c39da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:05.476506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4880b-58c4-4699-b004-743908c39da2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:05.476822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4880b-58c4-4699-b004-743908c39da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:05.477088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4880b-58c4-4699-b004-743908c39da2 None admin] PATH_INFO: `/v3/users/d0862144733c4764b4c5a003b52552c1` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:05.491419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4880b-58c4-4699-b004-743908c39da2 None admin] RBAC: Authorizing `identity:get_user(user_id=d0862144733c4764b4c5a003b52552c1)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:05.492909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4880b-58c4-4699-b004-743908c39da2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:05.495122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 514/1050] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:05 2022] GET /identity/v3/users/d0862144733c4764b4c5a003b52552c1 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:05.501028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6386f-2972-4337-8860-d32701238d00 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:05.514759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:05.614554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6386f-2972-4337-8860-d32701238d00 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:05.650474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387fe6b-059d-44ab-a1a9-298b85527877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:05.650998 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387fe6b-059d-44ab-a1a9-298b85527877 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:05.651273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387fe6b-059d-44ab-a1a9-298b85527877 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:05.651540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387fe6b-059d-44ab-a1a9-298b85527877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:05.721223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387fe6b-059d-44ab-a1a9-298b85527877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:05.722676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387fe6b-059d-44ab-a1a9-298b85527877 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:05.733925 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:05.829077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 537/1051] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:05.835161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4880b-58c4-4699-b004-743908c39da2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:05.944968 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:05.949803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4880b-58c4-4699-b004-743908c39da2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:06.012548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176ff20-edda-4862-84bd-62a20a8f06f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:06.013305 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176ff20-edda-4862-84bd-62a20a8f06f3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:06.013588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176ff20-edda-4862-84bd-62a20a8f06f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:06.013857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176ff20-edda-4862-84bd-62a20a8f06f3 None admin] PATH_INFO: `/v3/projects/ac563f47bac74778adfa5c58748b654b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:06.015420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176ff20-edda-4862-84bd-62a20a8f06f3 None admin] RBAC: Authorizing `identity:get_project(project_id=ac563f47bac74778adfa5c58748b654b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:06.017584 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176ff20-edda-4862-84bd-62a20a8f06f3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:06.019445 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 515/1052] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:05 2022] GET /identity/v3/projects/ac563f47bac74778adfa5c58748b654b => generated 311 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:06.026087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387fe6b-059d-44ab-a1a9-298b85527877 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:06.142924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387fe6b-059d-44ab-a1a9-298b85527877 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:06.156008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:06.199471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24724033-a17a-4298-b4f4-7baf3231cd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:06.199999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24724033-a17a-4298-b4f4-7baf3231cd7f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:06.200279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24724033-a17a-4298-b4f4-7baf3231cd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:06.200548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24724033-a17a-4298-b4f4-7baf3231cd7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:06.201327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24724033-a17a-4298-b4f4-7baf3231cd7f None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=ac563f47bac74778adfa5c58748b654b, user.id=d0862144733c4764b4c5a003b52552c1) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:06.201643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24724033-a17a-4298-b4f4-7baf3231cd7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:06.203087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24724033-a17a-4298-b4f4-7baf3231cd7f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:06.214048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8033d75-008f-43f4-a15f-0db05a9db5d4 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:06.214617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 538/1053] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:06 2022] GET /identity/v3/role_assignments?user.id=d0862144733c4764b4c5a003b52552c1&scope.project.id=ac563f47bac74778adfa5c58748b654b&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:06.214718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8033d75-008f-43f4-a15f-0db05a9db5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:06.214718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8033d75-008f-43f4-a15f-0db05a9db5d4 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:06.215885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 516/1054] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:06.224708 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa63e9-5746-4f66-80a9-f1beb003669e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:06.224708 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa63e9-5746-4f66-80a9-f1beb003669e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:06.224708 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa63e9-5746-4f66-80a9-f1beb003669e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:06.250967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b9aa63e9-5746-4f66-80a9-f1beb003669e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:06.252528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b9aa63e9-5746-4f66-80a9-f1beb003669e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:06.346018 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 539/1055] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:06.692623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8033d75-008f-43f4-a15f-0db05a9db5d4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:06.835601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8033d75-008f-43f4-a15f-0db05a9db5d4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:06.881822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7de52-8a80-4fd2-8edf-546f34b379af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:06.882820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7de52-8a80-4fd2-8edf-546f34b379af None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:06.883262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7de52-8a80-4fd2-8edf-546f34b379af None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:06.883709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7de52-8a80-4fd2-8edf-546f34b379af None admin] PATH_INFO: `/v3/services/image` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:06.884761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7de52-8a80-4fd2-8edf-546f34b379af None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:06.886436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7de52-8a80-4fd2-8edf-546f34b379af None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e2d7de52-8a80-4fd2-8edf-546f34b379af None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:06.897693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:06.898789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:21:06.899761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:06.904789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 517/1056] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Aug 24 12:21:06 2022] GET /identity/v3/services/image => generated 86 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:06.910622 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa63e9-5746-4f66-80a9-f1beb003669e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:06.983795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262c118-8152-4936-aa63-b7eb0f6ef305 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:06.983986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262c118-8152-4936-aa63-b7eb0f6ef305 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:06.984241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262c118-8152-4936-aa63-b7eb0f6ef305 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:06.988785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 518/1057] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:06.993725 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43937b-b682-4314-af5a-2b76150d89ab None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:06.994069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43937b-b682-4314-af5a-2b76150d89ab None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:06.994327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43937b-b682-4314-af5a-2b76150d89ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:07.015224 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7a43937b-b682-4314-af5a-2b76150d89ab None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:07.016778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7a43937b-b682-4314-af5a-2b76150d89ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:07.047437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa63e9-5746-4f66-80a9-f1beb003669e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:07.050910 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:07.052203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:07.090873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e586367-6e1e-4606-b75d-a061101ed00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:07.091647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e586367-6e1e-4606-b75d-a061101ed00c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:07.091985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e586367-6e1e-4606-b75d-a061101ed00c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:07.092315 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e586367-6e1e-4606-b75d-a061101ed00c None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:07.093060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e586367-6e1e-4606-b75d-a061101ed00c None admin] RBAC: Adding query filter params (name=image) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:07.093416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e586367-6e1e-4606-b75d-a061101ed00c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:07.094860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e586367-6e1e-4606-b75d-a061101ed00c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:07.104914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 540/1058] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:07.106289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e586367-6e1e-4606-b75d-a061101ed00c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:07.109014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 519/1059] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:21:07.210503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e586367-6e1e-4606-b75d-a061101ed00c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:07.245547 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9fcc8-588d-4a71-bbc0-c6aebd268740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:07.246075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9fcc8-588d-4a71-bbc0-c6aebd268740 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:07.246350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9fcc8-588d-4a71-bbc0-c6aebd268740 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:07.246615 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9fcc8-588d-4a71-bbc0-c6aebd268740 None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:07.247231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9fcc8-588d-4a71-bbc0-c6aebd268740 None admin] RBAC: Adding query filter params (type=image) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:07.247527 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9fcc8-588d-4a71-bbc0-c6aebd268740 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:07.248903 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9fcc8-588d-4a71-bbc0-c6aebd268740 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:07.253933 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 541/1060] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:07.259754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43937b-b682-4314-af5a-2b76150d89ab None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:07.364994 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43937b-b682-4314-af5a-2b76150d89ab None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:07.374086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9fcc8-588d-4a71-bbc0-c6aebd268740 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:07.403935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad6a8d-a3a2-4dd5-b0bc-97280800dc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:07.404557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad6a8d-a3a2-4dd5-b0bc-97280800dc67 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:07.404866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad6a8d-a3a2-4dd5-b0bc-97280800dc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:07.405138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad6a8d-a3a2-4dd5-b0bc-97280800dc67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:07.405846 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad6a8d-a3a2-4dd5-b0bc-97280800dc67 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:07.407288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad6a8d-a3a2-4dd5-b0bc-97280800dc67 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:07.409194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:07.436683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 520/1061] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:21:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:07.517430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9fcc8-588d-4a71-bbc0-c6aebd268740 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:07.562211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53c4ac-9752-45d0-b354-80a76ac723df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:07.562859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53c4ac-9752-45d0-b354-80a76ac723df None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:07.563134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53c4ac-9752-45d0-b354-80a76ac723df None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:07.563403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53c4ac-9752-45d0-b354-80a76ac723df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:07.575463 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53c4ac-9752-45d0-b354-80a76ac723df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3d53c4ac-9752-45d0-b354-80a76ac723df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:07.580123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:07.581085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:07.581943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 542/1062] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 24 12:21:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:21:07.588081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad6a8d-a3a2-4dd5-b0bc-97280800dc67 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:07.610728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:07.698979 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad6a8d-a3a2-4dd5-b0bc-97280800dc67 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:07.735162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a04cc-f796-4aae-a981-1307eb65480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:07.735772 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a04cc-f796-4aae-a981-1307eb65480b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:07.736049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a04cc-f796-4aae-a981-1307eb65480b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:07.736316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a04cc-f796-4aae-a981-1307eb65480b None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:07.737047 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a04cc-f796-4aae-a981-1307eb65480b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:07.737349 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a04cc-f796-4aae-a981-1307eb65480b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:07.738728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a04cc-f796-4aae-a981-1307eb65480b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:07.747208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 521/1063] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 12:21:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:07.752968 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53c4ac-9752-45d0-b354-80a76ac723df None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:07.845796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:07.859567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53c4ac-9752-45d0-b354-80a76ac723df None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:07.895011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3077fa5-2d2a-40c5-a721-a791babe4336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:07.895639 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3077fa5-2d2a-40c5-a721-a791babe4336 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:07.895912 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3077fa5-2d2a-40c5-a721-a791babe4336 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:07.896180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3077fa5-2d2a-40c5-a721-a791babe4336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:07.907233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3077fa5-2d2a-40c5-a721-a791babe4336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:07.908711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3077fa5-2d2a-40c5-a721-a791babe4336 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-d3077fa5-2d2a-40c5-a721-a791babe4336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:07.922984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:07.923969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:21:07.924883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:07.924883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:07.924883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:07.928732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 543/1064] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:07.930380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a04cc-f796-4aae-a981-1307eb65480b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:08.025834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:08.047142 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a04cc-f796-4aae-a981-1307eb65480b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:08.081773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ab34b-8ebd-4a9c-a0cc-697259f93605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:08.082381 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ab34b-8ebd-4a9c-a0cc-697259f93605 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.082655 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ab34b-8ebd-4a9c-a0cc-697259f93605 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.082923 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ab34b-8ebd-4a9c-a0cc-697259f93605 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.083564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ab34b-8ebd-4a9c-a0cc-697259f93605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:08.083860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ab34b-8ebd-4a9c-a0cc-697259f93605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:08.085289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ab34b-8ebd-4a9c-a0cc-697259f93605 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:08.103314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 522/1065] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:08.104754 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3077fa5-2d2a-40c5-a721-a791babe4336 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:08.210423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3077fa5-2d2a-40c5-a721-a791babe4336 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:08.244440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413bf72-e9da-4627-9c59-565c4ecb8127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:08.245108 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413bf72-e9da-4627-9c59-565c4ecb8127 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.245346 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413bf72-e9da-4627-9c59-565c4ecb8127 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.245614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413bf72-e9da-4627-9c59-565c4ecb8127 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.247942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152ee32-1e21-4810-a2bc-d37d282bbeee None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.248227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152ee32-1e21-4810-a2bc-d37d282bbeee None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.248490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152ee32-1e21-4810-a2bc-d37d282bbeee None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.250059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 523/1066] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:08.253871 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413bf72-e9da-4627-9c59-565c4ecb8127 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:08.254747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deb7e2-7e4b-48ab-858d-dc8e570464b1 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.255030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deb7e2-7e4b-48ab-858d-dc8e570464b1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.255303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deb7e2-7e4b-48ab-858d-dc8e570464b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.255398 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413bf72-e9da-4627-9c59-565c4ecb8127 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:08.258508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-0413bf72-e9da-4627-9c59-565c4ecb8127 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:21:08.269236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:08.270211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:08.276329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:08.276329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:08.276329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:21:08.276329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:21:08.276329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:21:08.276329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:21:08.276329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:21:08.276329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:21:08.276329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:21:08.276329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:08.276329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 544/1067] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 12:21:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:08.277099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-99deb7e2-7e4b-48ab-858d-dc8e570464b1 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:08.277838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-99deb7e2-7e4b-48ab-858d-dc8e570464b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:08.278904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413bf72-e9da-4627-9c59-565c4ecb8127 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:08.349882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 524/1068] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:21:08.388342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413bf72-e9da-4627-9c59-565c4ecb8127 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:08.423458 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd65767-d738-49d2-bc0a-eedbef4fdf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:08.424066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd65767-d738-49d2-bc0a-eedbef4fdf5b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.424339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd65767-d738-49d2-bc0a-eedbef4fdf5b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.424607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd65767-d738-49d2-bc0a-eedbef4fdf5b None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.425365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd65767-d738-49d2-bc0a-eedbef4fdf5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:08.425662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd65767-d738-49d2-bc0a-eedbef4fdf5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:08.427092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd65767-d738-49d2-bc0a-eedbef4fdf5b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:08.448285 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 545/1069] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 12:21:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:08.450926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deb7e2-7e4b-48ab-858d-dc8e570464b1 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:08.523791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:08.559911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deb7e2-7e4b-48ab-858d-dc8e570464b1 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:08.595889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d913cd-7832-49ac-8723-66488dcc6372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:08.596518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d913cd-7832-49ac-8723-66488dcc6372 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.596822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d913cd-7832-49ac-8723-66488dcc6372 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.597094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d913cd-7832-49ac-8723-66488dcc6372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.607815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d913cd-7832-49ac-8723-66488dcc6372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:08.609306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d913cd-7832-49ac-8723-66488dcc6372 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:08.618857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd65767-d738-49d2-bc0a-eedbef4fdf5b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-27d913cd-7832-49ac-8723-66488dcc6372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:08.624332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:08.625456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:21:08.626345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:08.626345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:08.626345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:08.626982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 525/1070] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:08.631844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d913cd-7832-49ac-8723-66488dcc6372 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:08.685360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:08.733529 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd65767-d738-49d2-bc0a-eedbef4fdf5b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:08.741804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d913cd-7832-49ac-8723-66488dcc6372 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:08.767874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:08.772800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3a946-f44b-4de2-b96a-ee269bef83ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:08.773438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3a946-f44b-4de2-b96a-ee269bef83ba None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.773766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3a946-f44b-4de2-b96a-ee269bef83ba None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.774032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3a946-f44b-4de2-b96a-ee269bef83ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.781005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b20d0-25b2-4518-a11a-3a8c748159d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:08.781863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b20d0-25b2-4518-a11a-3a8c748159d4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.782330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b20d0-25b2-4518-a11a-3a8c748159d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.782671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b20d0-25b2-4518-a11a-3a8c748159d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.783323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b20d0-25b2-4518-a11a-3a8c748159d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:08.783626 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b20d0-25b2-4518-a11a-3a8c748159d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:08.785037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b20d0-25b2-4518-a11a-3a8c748159d4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:08.785143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3a946-f44b-4de2-b96a-ee269bef83ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-73a3a946-f44b-4de2-b96a-ee269bef83ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:08.789640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:08.790601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:08.791700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 546/1071] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:21:08.796862 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3a946-f44b-4de2-b96a-ee269bef83ba None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:08.803673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 526/1072] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:08.809530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b20d0-25b2-4518-a11a-3a8c748159d4 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:08.903977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3a946-f44b-4de2-b96a-ee269bef83ba None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:08.911927 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b20d0-25b2-4518-a11a-3a8c748159d4 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:08.923773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:08.943860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0cbf08-2aad-493e-94ca-1a2208434230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:08.944486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cbf08-2aad-493e-94ca-1a2208434230 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.944869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cbf08-2aad-493e-94ca-1a2208434230 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.945143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cbf08-2aad-493e-94ca-1a2208434230 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.945810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0cbf08-2aad-493e-94ca-1a2208434230 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:08.946208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0cbf08-2aad-493e-94ca-1a2208434230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:08.946586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4da79-c910-47d4-873e-92650cb21115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:08.948133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4da79-c910-47d4-873e-92650cb21115 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.948133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4da79-c910-47d4-873e-92650cb21115 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.948332 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0cbf08-2aad-493e-94ca-1a2208434230 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:08.948405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4da79-c910-47d4-873e-92650cb21115 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.954868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4da79-c910-47d4-873e-92650cb21115 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:08.956314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4da79-c910-47d4-873e-92650cb21115 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:08.960852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 547/1073] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:08.963191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0cbf08-2aad-493e-94ca-1a2208434230 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-36f4da79-c910-47d4-873e-92650cb21115 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:21:08.973013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:08.973940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:08.980033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:21:08.980033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:21:08.980033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:21:08.980033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:21:08.980033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:21:08.980033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:21:08.980033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:21:08.980033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:08.980033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 527/1074] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:21:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:21:08.981932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4da79-c910-47d4-873e-92650cb21115 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:09.029260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:09.078770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0cbf08-2aad-493e-94ca-1a2208434230 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:09.091623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4da79-c910-47d4-873e-92650cb21115 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:09.112583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419d81c-61ac-4b60-aa60-c329b994de2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:09.113247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419d81c-61ac-4b60-aa60-c329b994de2e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:09.113520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419d81c-61ac-4b60-aa60-c329b994de2e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:09.113784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419d81c-61ac-4b60-aa60-c329b994de2e None admin] PATH_INFO: `/v3/users/d0862144733c4764b4c5a003b52552c1` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:09.114898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419d81c-61ac-4b60-aa60-c329b994de2e None admin] RBAC: Authorizing `identity:get_user(user_id=d0862144733c4764b4c5a003b52552c1)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:09.116330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419d81c-61ac-4b60-aa60-c329b994de2e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:09.117939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 548/1075] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:08 2022] GET /identity/v3/users/d0862144733c4764b4c5a003b52552c1 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:09.123634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419d81c-61ac-4b60-aa60-c329b994de2e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:09.133966 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac4c7a-965e-486d-a111-a565bf374d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:09.134665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac4c7a-965e-486d-a111-a565bf374d93 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:09.134945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac4c7a-965e-486d-a111-a565bf374d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:09.135217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac4c7a-965e-486d-a111-a565bf374d93 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:09.135920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac4c7a-965e-486d-a111-a565bf374d93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:09.136221 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac4c7a-965e-486d-a111-a565bf374d93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:09.137708 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac4c7a-965e-486d-a111-a565bf374d93 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:09.155192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 528/1076] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:21:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:09.156527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac4c7a-965e-486d-a111-a565bf374d93 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:09.178917 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:09.236898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419d81c-61ac-4b60-aa60-c329b994de2e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:09.267958 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac4c7a-965e-486d-a111-a565bf374d93 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:09.269859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:09.275969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0836289-689d-40d4-ab2b-b380a70595a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:09.276643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0836289-689d-40d4-ab2b-b380a70595a6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:09.276929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0836289-689d-40d4-ab2b-b380a70595a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:09.277202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0836289-689d-40d4-ab2b-b380a70595a6 None admin] PATH_INFO: `/v3/projects/ac563f47bac74778adfa5c58748b654b` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:09.278422 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0836289-689d-40d4-ab2b-b380a70595a6 None admin] RBAC: Authorizing `identity:get_project(project_id=ac563f47bac74778adfa5c58748b654b)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:09.279883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0836289-689d-40d4-ab2b-b380a70595a6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:09.281749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 549/1077] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:09 2022] GET /identity/v3/projects/ac563f47bac74778adfa5c58748b654b => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:09.287723 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0836289-689d-40d4-ab2b-b380a70595a6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:09.309430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad563cbc-cce1-4e54-9354-d02f8771329e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:09.310108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad563cbc-cce1-4e54-9354-d02f8771329e None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:09.310367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad563cbc-cce1-4e54-9354-d02f8771329e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:09.310634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad563cbc-cce1-4e54-9354-d02f8771329e None admin] PATH_INFO: `/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/e65026e01df242e7be11ed24214a4efa/roles/1b4412853cfe43d3b524d26be06750ef` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:09.312818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad563cbc-cce1-4e54-9354-d02f8771329e None admin] RBAC: Authorizing `identity:create_grant(project_id=70a5f9cf0499405d82f5f5a47c2877eb, user_id=e65026e01df242e7be11ed24214a4efa, role_id=1b4412853cfe43d3b524d26be06750ef)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:09.314278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad563cbc-cce1-4e54-9354-d02f8771329e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:09.329711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 529/1078] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:21:09 2022] PUT /identity/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/e65026e01df242e7be11ed24214a4efa/roles/1b4412853cfe43d3b524d26be06750ef => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:21:09.378517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab62ba3-629c-4103-a8f8-765c4bd56b57 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:09.378786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab62ba3-629c-4103-a8f8-765c4bd56b57 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:09.379055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab62ba3-629c-4103-a8f8-765c4bd56b57 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:09.380792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 530/1079] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:21:09.387651 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5fc85-386f-4a7d-85f8-51cbd4339f75 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:09.387964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5fc85-386f-4a7d-85f8-51cbd4339f75 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:09.388362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5fc85-386f-4a7d-85f8-51cbd4339f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:09.408566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-81f5fc85-386f-4a7d-85f8-51cbd4339f75 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:09.409920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-81f5fc85-386f-4a7d-85f8-51cbd4339f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:09.425907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0836289-689d-40d4-ab2b-b380a70595a6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:09.445873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:09.468331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5eb7c-bc97-4d88-b25c-954bfaa4ede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:09.469218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5eb7c-bc97-4d88-b25c-954bfaa4ede6 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:09.469604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5eb7c-bc97-4d88-b25c-954bfaa4ede6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:09.469941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5eb7c-bc97-4d88-b25c-954bfaa4ede6 None admin] PATH_INFO: `/v3/projects/ac563f47bac74778adfa5c58748b654b/users/d0862144733c4764b4c5a003b52552c1/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:09.472164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5eb7c-bc97-4d88-b25c-954bfaa4ede6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac563f47bac74778adfa5c58748b654b, user_id=d0862144733c4764b4c5a003b52552c1, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:09.473853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5eb7c-bc97-4d88-b25c-954bfaa4ede6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:09.492251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 550/1080] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:21:09 2022] PUT /identity/v3/projects/ac563f47bac74778adfa5c58748b654b/users/d0862144733c4764b4c5a003b52552c1/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:21:09.499049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 531/1081] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:09.775911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5eb7c-bc97-4d88-b25c-954bfaa4ede6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:09.881888 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5eb7c-bc97-4d88-b25c-954bfaa4ede6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:09.918216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e863a4-e4c9-475b-a33c-72eb93bfd87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:09.918828 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e863a4-e4c9-475b-a33c-72eb93bfd87b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:09.919104 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e863a4-e4c9-475b-a33c-72eb93bfd87b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:09.919371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e863a4-e4c9-475b-a33c-72eb93bfd87b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:09.970081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:09.998128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e863a4-e4c9-475b-a33c-72eb93bfd87b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:09.999590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e863a4-e4c9-475b-a33c-72eb93bfd87b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:10.099459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 551/1082] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:10.105273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5fc85-386f-4a7d-85f8-51cbd4339f75 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:10.196928 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:10.208623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5fc85-386f-4a7d-85f8-51cbd4339f75 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:10.243604 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45403d-8c13-4138-bf31-5d26ee1548dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:10.244424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45403d-8c13-4138-bf31-5d26ee1548dd None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:10.244789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45403d-8c13-4138-bf31-5d26ee1548dd None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:10.245119 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45403d-8c13-4138-bf31-5d26ee1548dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:10.255369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45403d-8c13-4138-bf31-5d26ee1548dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:10.256917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45403d-8c13-4138-bf31-5d26ee1548dd None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-8a45403d-8c13-4138-bf31-5d26ee1548dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:10.271603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:10.272603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:21:10.273831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:10.273831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:10.273831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:10.274575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 532/1083] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:10.279090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e863a4-e4c9-475b-a33c-72eb93bfd87b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:10.369134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:10.395286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e863a4-e4c9-475b-a33c-72eb93bfd87b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:10.434716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133757a-c652-4a75-b9d1-b068caa4b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:10.435334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133757a-c652-4a75-b9d1-b068caa4b4dd None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:10.435614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133757a-c652-4a75-b9d1-b068caa4b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:10.435885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133757a-c652-4a75-b9d1-b068caa4b4dd None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:10.436547 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133757a-c652-4a75-b9d1-b068caa4b4dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:10.436869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133757a-c652-4a75-b9d1-b068caa4b4dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:10.438242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133757a-c652-4a75-b9d1-b068caa4b4dd None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:10.450375 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 552/1084] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:11.248086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3b5b3-a0a9-4420-a72a-830a2bd0269d None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:11.248615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3b5b3-a0a9-4420-a72a-830a2bd0269d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:11.248615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3b5b3-a0a9-4420-a72a-830a2bd0269d None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:11.250041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 533/1085] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:11.255739 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29561062-da11-46bb-a71f-6f5c5f7d0c5e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:11.256027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29561062-da11-46bb-a71f-6f5c5f7d0c5e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:11.256299 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29561062-da11-46bb-a71f-6f5c5f7d0c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:11.275926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-29561062-da11-46bb-a71f-6f5c5f7d0c5e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:11.277146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-29561062-da11-46bb-a71f-6f5c5f7d0c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:11.348521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 553/1086] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:11.430317 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3702cd-8784-4234-aeec-3846258528d6 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:11.430564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3702cd-8784-4234-aeec-3846258528d6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:11.430868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3702cd-8784-4234-aeec-3846258528d6 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:11.432982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 534/1087] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:11.438188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a784007-faf8-4d97-b11b-9eac39a83c26 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:11.438477 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a784007-faf8-4d97-b11b-9eac39a83c26 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:11.438752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a784007-faf8-4d97-b11b-9eac39a83c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:11.458242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2a784007-faf8-4d97-b11b-9eac39a83c26 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:11.459429 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2a784007-faf8-4d97-b11b-9eac39a83c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:11.534277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:11.544339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 554/1088] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:11.623673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3702cd-8784-4234-aeec-3846258528d6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:11.731873 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3702cd-8784-4234-aeec-3846258528d6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:11.770795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55e621-cf67-43b0-a217-04d5adcb5cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:11.771422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55e621-cf67-43b0-a217-04d5adcb5cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:11.771694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55e621-cf67-43b0-a217-04d5adcb5cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:11.771957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55e621-cf67-43b0-a217-04d5adcb5cc6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:11.782934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55e621-cf67-43b0-a217-04d5adcb5cc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-1d55e621-cf67-43b0-a217-04d5adcb5cc6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:11.787698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:11.788637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:11.789508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 535/1089] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 24 12:21:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:11.795008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a784007-faf8-4d97-b11b-9eac39a83c26 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:11.831279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55e621-cf67-43b0-a217-04d5adcb5cc6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:11.837608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:11.897962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a784007-faf8-4d97-b11b-9eac39a83c26 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:11.933851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea16ae-e165-4b65-a30b-347a5c2ea884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:11.934496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea16ae-e165-4b65-a30b-347a5c2ea884 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:11.934783 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea16ae-e165-4b65-a30b-347a5c2ea884 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:11.935063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea16ae-e165-4b65-a30b-347a5c2ea884 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:11.935773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea16ae-e165-4b65-a30b-347a5c2ea884 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:11.936070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea16ae-e165-4b65-a30b-347a5c2ea884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:11.936669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55e621-cf67-43b0-a217-04d5adcb5cc6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:11.937489 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea16ae-e165-4b65-a30b-347a5c2ea884 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:11.945584 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 555/1090] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 12:21:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:11.951575 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea16ae-e165-4b65-a30b-347a5c2ea884 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:11.977907 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cb62d-397d-48a7-89c1-ebf51c1986e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:11.978643 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cb62d-397d-48a7-89c1-ebf51c1986e7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:11.979360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cb62d-397d-48a7-89c1-ebf51c1986e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:11.979360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cb62d-397d-48a7-89c1-ebf51c1986e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:11.990439 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cb62d-397d-48a7-89c1-ebf51c1986e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b66cb62d-397d-48a7-89c1-ebf51c1986e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:11.995101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:11.996076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:11.996935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 536/1091] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:12.002325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cb62d-397d-48a7-89c1-ebf51c1986e7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:12.016791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:12.060690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea16ae-e165-4b65-a30b-347a5c2ea884 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:12.064826 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:12.095346 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670fcb2-ed56-4a2e-9ac9-a90347e22256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:12.095966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670fcb2-ed56-4a2e-9ac9-a90347e22256 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:12.096238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670fcb2-ed56-4a2e-9ac9-a90347e22256 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:12.096501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670fcb2-ed56-4a2e-9ac9-a90347e22256 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:12.103443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cb62d-397d-48a7-89c1-ebf51c1986e7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:12.107125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670fcb2-ed56-4a2e-9ac9-a90347e22256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:12.108568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670fcb2-ed56-4a2e-9ac9-a90347e22256 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-9670fcb2-ed56-4a2e-9ac9-a90347e22256 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:12.122164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:12.123334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:21:12.124241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:12.124241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:12.124241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:12.124241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 556/1092] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:21:12.129299 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670fcb2-ed56-4a2e-9ac9-a90347e22256 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:12.142283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0073a-2d22-439f-92da-d06ccc882685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:12.142888 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0073a-2d22-439f-92da-d06ccc882685 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:12.143161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0073a-2d22-439f-92da-d06ccc882685 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:12.143428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0073a-2d22-439f-92da-d06ccc882685 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:12.144086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0073a-2d22-439f-92da-d06ccc882685 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:12.144382 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0073a-2d22-439f-92da-d06ccc882685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:12.145792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0073a-2d22-439f-92da-d06ccc882685 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:12.155118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 537/1093] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:12.161029 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0073a-2d22-439f-92da-d06ccc882685 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:12.228615 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670fcb2-ed56-4a2e-9ac9-a90347e22256 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:12.239026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:12.262226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0073a-2d22-439f-92da-d06ccc882685 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:12.262894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd74da4-7f49-4bc3-957f-03c631d5dd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:12.263498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd74da4-7f49-4bc3-957f-03c631d5dd45 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:12.263775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd74da4-7f49-4bc3-957f-03c631d5dd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:12.264039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd74da4-7f49-4bc3-957f-03c631d5dd45 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:12.265152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd74da4-7f49-4bc3-957f-03c631d5dd45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:12.265152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd74da4-7f49-4bc3-957f-03c631d5dd45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:12.266342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd74da4-7f49-4bc3-957f-03c631d5dd45 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:12.277901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 557/1094] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:12.283771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd74da4-7f49-4bc3-957f-03c631d5dd45 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:12.288707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:12.298728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccaa4e-9764-48a0-ba30-6c079f501491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:12.298728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccaa4e-9764-48a0-ba30-6c079f501491 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:12.298954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccaa4e-9764-48a0-ba30-6c079f501491 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:12.299104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccaa4e-9764-48a0-ba30-6c079f501491 None admin] PATH_INFO: `/v3/users/d0862144733c4764b4c5a003b52552c1` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:12.300237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccaa4e-9764-48a0-ba30-6c079f501491 None admin] RBAC: Authorizing `identity:get_user(user_id=d0862144733c4764b4c5a003b52552c1)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:12.301687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccaa4e-9764-48a0-ba30-6c079f501491 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:12.303307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 538/1095] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:12 2022] GET /identity/v3/users/d0862144733c4764b4c5a003b52552c1 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:12.309194 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccaa4e-9764-48a0-ba30-6c079f501491 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:12.384962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd74da4-7f49-4bc3-957f-03c631d5dd45 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:12.409939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccaa4e-9764-48a0-ba30-6c079f501491 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:12.419333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d4b1d-3d02-4d4b-abde-6395c1d727ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:12.419961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d4b1d-3d02-4d4b-abde-6395c1d727ac None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:12.420234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d4b1d-3d02-4d4b-abde-6395c1d727ac None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:12.420496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d4b1d-3d02-4d4b-abde-6395c1d727ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:12.427456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d4b1d-3d02-4d4b-abde-6395c1d727ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:12.428927 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d4b1d-3d02-4d4b-abde-6395c1d727ac None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-848d4b1d-3d02-4d4b-abde-6395c1d727ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:21:12.440153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:12.441288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:12.442624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:12.442624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:12.442624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:21:12.442624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:21:12.442624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:21:12.442624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:21:12.442624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:21:12.442624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:21:12.442624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:21:12.442624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:12.442624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 558/1096] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 12:21:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:21:12.443846 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23155f-2e35-4ea1-be79-7abcf2025717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:12.444451 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23155f-2e35-4ea1-be79-7abcf2025717 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:12.444757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23155f-2e35-4ea1-be79-7abcf2025717 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:12.445027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23155f-2e35-4ea1-be79-7abcf2025717 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:12.447012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d4b1d-3d02-4d4b-abde-6395c1d727ac None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:12.478426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:12.510061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:12.518416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23155f-2e35-4ea1-be79-7abcf2025717 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:12.519882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23155f-2e35-4ea1-be79-7abcf2025717 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:12.551068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d4b1d-3d02-4d4b-abde-6395c1d727ac None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:12.585057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75a70e-397e-4378-aed0-2a4b899bfa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:12.585659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75a70e-397e-4378-aed0-2a4b899bfa00 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:12.585930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75a70e-397e-4378-aed0-2a4b899bfa00 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:12.586542 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75a70e-397e-4378-aed0-2a4b899bfa00 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:12.587403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75a70e-397e-4378-aed0-2a4b899bfa00 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:12.587916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75a70e-397e-4378-aed0-2a4b899bfa00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:12.589263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75a70e-397e-4378-aed0-2a4b899bfa00 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:12.608822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 559/1097] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 12:21:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:12.615530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75a70e-397e-4378-aed0-2a4b899bfa00 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:12.627456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 539/1098] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 12:21:12.633290 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23155f-2e35-4ea1-be79-7abcf2025717 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:12.709941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:12.723444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75a70e-397e-4378-aed0-2a4b899bfa00 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:12.732745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:12.739993 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23155f-2e35-4ea1-be79-7abcf2025717 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:12.756802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b720c9-980c-4f8f-9047-47ee94e327ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:12.757409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b720c9-980c-4f8f-9047-47ee94e327ac None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:12.757684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b720c9-980c-4f8f-9047-47ee94e327ac None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:12.757960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b720c9-980c-4f8f-9047-47ee94e327ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:12.774891 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71802f-2df0-4d4f-9fd8-24536ee99fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:12.775514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71802f-2df0-4d4f-9fd8-24536ee99fff None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:12.775834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71802f-2df0-4d4f-9fd8-24536ee99fff None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:12.776056 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71802f-2df0-4d4f-9fd8-24536ee99fff None admin] PATH_INFO: `/v3/projects/ac563f47bac74778adfa5c58748b654b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:12.779050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71802f-2df0-4d4f-9fd8-24536ee99fff None admin] RBAC: Authorizing `identity:get_project(project_id=ac563f47bac74778adfa5c58748b654b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:12.780496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71802f-2df0-4d4f-9fd8-24536ee99fff None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:12.783237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 540/1099] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:12 2022] GET /identity/v3/projects/ac563f47bac74778adfa5c58748b654b => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:12.788506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71802f-2df0-4d4f-9fd8-24536ee99fff None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:12.826815 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b720c9-980c-4f8f-9047-47ee94e327ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:12.828274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b720c9-980c-4f8f-9047-47ee94e327ac None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:12.888692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71802f-2df0-4d4f-9fd8-24536ee99fff None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:12.914634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:12.923120 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a725a0-be4c-4480-afac-06f5ee934288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:12.923715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a725a0-be4c-4480-afac-06f5ee934288 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:12.923987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a725a0-be4c-4480-afac-06f5ee934288 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:12.924251 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a725a0-be4c-4480-afac-06f5ee934288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:12.924997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a725a0-be4c-4480-afac-06f5ee934288 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=ac563f47bac74778adfa5c58748b654b, user.id=d0862144733c4764b4c5a003b52552c1) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:12.925321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a725a0-be4c-4480-afac-06f5ee934288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:12.926738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a725a0-be4c-4480-afac-06f5ee934288 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:12.932848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 560/1100] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:12.932984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 541/1101] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:12 2022] GET /identity/v3/role_assignments?user.id=d0862144733c4764b4c5a003b52552c1&scope.project.id=ac563f47bac74778adfa5c58748b654b&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:12.937879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b720c9-980c-4f8f-9047-47ee94e327ac None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:13.055134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b720c9-980c-4f8f-9047-47ee94e327ac None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:13.094789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb73803-154f-4fbe-82af-f0e284ff8e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:13.095422 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb73803-154f-4fbe-82af-f0e284ff8e9e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:13.095697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb73803-154f-4fbe-82af-f0e284ff8e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:13.095970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb73803-154f-4fbe-82af-f0e284ff8e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:13.106236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb73803-154f-4fbe-82af-f0e284ff8e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:13.107688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb73803-154f-4fbe-82af-f0e284ff8e9e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-5cb73803-154f-4fbe-82af-f0e284ff8e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:13.122712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:13.123852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:21:13.125080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:13.125080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:13.125080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:13.125080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 561/1102] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:21:13.129819 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a725a0-be4c-4480-afac-06f5ee934288 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:13.157945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:13.230238 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a725a0-be4c-4480-afac-06f5ee934288 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:13.264995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d416e-ce22-437a-a685-ae8a3cb80899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:13.265713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d416e-ce22-437a-a685-ae8a3cb80899 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:13.265808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d416e-ce22-437a-a685-ae8a3cb80899 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:13.266107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d416e-ce22-437a-a685-ae8a3cb80899 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:13.266749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d416e-ce22-437a-a685-ae8a3cb80899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:13.267050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d416e-ce22-437a-a685-ae8a3cb80899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:13.268430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d416e-ce22-437a-a685-ae8a3cb80899 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:13.281011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 542/1103] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:13.286798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb73803-154f-4fbe-82af-f0e284ff8e9e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:13.329531 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:13.392376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb73803-154f-4fbe-82af-f0e284ff8e9e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:13.427793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc4104-ceed-4c36-afeb-5eadcfb47320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:13.428414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc4104-ceed-4c36-afeb-5eadcfb47320 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:13.428681 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc4104-ceed-4c36-afeb-5eadcfb47320 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:13.428974 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc4104-ceed-4c36-afeb-5eadcfb47320 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:13.435663 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc4104-ceed-4c36-afeb-5eadcfb47320 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:13.437144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc4104-ceed-4c36-afeb-5eadcfb47320 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-f4fc4104-ceed-4c36-afeb-5eadcfb47320 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:21:13.448135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:13.449094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:13.455134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:21:13.455134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:21:13.455134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:21:13.455134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:21:13.455134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:21:13.455134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:21:13.455134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:21:13.455134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:13.455134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 562/1104] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:21:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:13.455633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d416e-ce22-437a-a685-ae8a3cb80899 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:13.542818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:13.560537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d416e-ce22-437a-a685-ae8a3cb80899 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:13.598467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf95647-7acd-4d66-98a5-a853dba32b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:13.599110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf95647-7acd-4d66-98a5-a853dba32b83 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:13.599390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf95647-7acd-4d66-98a5-a853dba32b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:13.599657 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf95647-7acd-4d66-98a5-a853dba32b83 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:13.600354 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf95647-7acd-4d66-98a5-a853dba32b83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:13.600681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf95647-7acd-4d66-98a5-a853dba32b83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:13.602230 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf95647-7acd-4d66-98a5-a853dba32b83 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:13.615688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 543/1105] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:21:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:13.622687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc4104-ceed-4c36-afeb-5eadcfb47320 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:13.724771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:13.728205 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc4104-ceed-4c36-afeb-5eadcfb47320 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:13.763442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc6659-3035-4690-854a-bbccaeacac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:13.764032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc6659-3035-4690-854a-bbccaeacac4d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:13.764304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc6659-3035-4690-854a-bbccaeacac4d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:13.764564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc6659-3035-4690-854a-bbccaeacac4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:13.765302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc6659-3035-4690-854a-bbccaeacac4d None admin] RBAC: Adding query filter params (role.id=1b4412853cfe43d3b524d26be06750ef, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=e65026e01df242e7be11ed24214a4efa) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:13.765603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc6659-3035-4690-854a-bbccaeacac4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:13.767030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc6659-3035-4690-854a-bbccaeacac4d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:13.772973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 563/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:13 2022] GET /identity/v3/role_assignments?user.id=e65026e01df242e7be11ed24214a4efa&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=1b4412853cfe43d3b524d26be06750ef => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:14.862216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc05176-0305-460f-830b-842b482473a7 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:14.863116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc05176-0305-460f-830b-842b482473a7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:14.863116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc05176-0305-460f-830b-842b482473a7 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:14.865093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 544/1107] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:14.872878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6f2df-4a5c-4109-a64b-f0816a0fe446 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:14.873338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6f2df-4a5c-4109-a64b-f0816a0fe446 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:14.873448 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6f2df-4a5c-4109-a64b-f0816a0fe446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:14.901319 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-52e6f2df-4a5c-4109-a64b-f0816a0fe446 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:14.902499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-52e6f2df-4a5c-4109-a64b-f0816a0fe446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:14.983868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 564/1108] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:15.256720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc05176-0305-460f-830b-842b482473a7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:15.373536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc05176-0305-460f-830b-842b482473a7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:15.414096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87b68f-a351-47c7-8f3a-27694eb73748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:15.414720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87b68f-a351-47c7-8f3a-27694eb73748 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:15.414996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87b68f-a351-47c7-8f3a-27694eb73748 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:15.415316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87b68f-a351-47c7-8f3a-27694eb73748 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:15.416564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87b68f-a351-47c7-8f3a-27694eb73748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:15.418032 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87b68f-a351-47c7-8f3a-27694eb73748 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:15.419639 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 545/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:15.425945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6f2df-4a5c-4109-a64b-f0816a0fe446 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:15.523596 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:15.534600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6f2df-4a5c-4109-a64b-f0816a0fe446 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:15.579133 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14436700-043d-40d6-8a66-63ef4b282992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:15.579734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14436700-043d-40d6-8a66-63ef4b282992 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:15.580007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14436700-043d-40d6-8a66-63ef4b282992 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:15.580274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14436700-043d-40d6-8a66-63ef4b282992 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:15.580969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14436700-043d-40d6-8a66-63ef4b282992 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:15.582440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14436700-043d-40d6-8a66-63ef4b282992 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:15.613386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 565/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:21:15 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:17.550619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccbec5-48bd-4a80-a48b-bdaf166970fd None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:17.551528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccbec5-48bd-4a80-a48b-bdaf166970fd None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:17.551528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccbec5-48bd-4a80-a48b-bdaf166970fd None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:17.553793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 546/1111] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:17.559555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f08a3c-7b49-43cc-865f-fccb7a789975 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:17.559996 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f08a3c-7b49-43cc-865f-fccb7a789975 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:17.560107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f08a3c-7b49-43cc-865f-fccb7a789975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:17.579942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b4f08a3c-7b49-43cc-865f-fccb7a789975 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:17.581150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b4f08a3c-7b49-43cc-865f-fccb7a789975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:17.651586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 566/1112] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:17.916561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccbec5-48bd-4a80-a48b-bdaf166970fd None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:17.939804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:18.033704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccbec5-48bd-4a80-a48b-bdaf166970fd None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:18.077977 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41cd42-aaec-449b-bbc8-48461bb9ef71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:18.078597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41cd42-aaec-449b-bbc8-48461bb9ef71 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:18.078866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41cd42-aaec-449b-bbc8-48461bb9ef71 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:18.079145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41cd42-aaec-449b-bbc8-48461bb9ef71 None admin] PATH_INFO: `/v3/roles/16317a22e9c94a8886f02efb75ab456e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:18.080681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41cd42-aaec-449b-bbc8-48461bb9ef71 None admin] RBAC: Authorizing `identity:get_role(role_id=16317a22e9c94a8886f02efb75ab456e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:18.082214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41cd42-aaec-449b-bbc8-48461bb9ef71 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:18.083648 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 547/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:17 2022] GET /identity/v3/roles/16317a22e9c94a8886f02efb75ab456e => generated 223 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:18.090908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f08a3c-7b49-43cc-865f-fccb7a789975 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:18.180020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:18.199943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f08a3c-7b49-43cc-865f-fccb7a789975 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:18.233810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4f026-da31-401e-a4c2-541be3b53d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:18.234426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4f026-da31-401e-a4c2-541be3b53d64 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:18.234702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4f026-da31-401e-a4c2-541be3b53d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:18.234977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4f026-da31-401e-a4c2-541be3b53d64 None admin] PATH_INFO: `/v3/users/b83eed3cde624d0a8f375a0ea53fed49` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:18.248718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4f026-da31-401e-a4c2-541be3b53d64 None admin] RBAC: Authorizing `identity:get_user(user_id=b83eed3cde624d0a8f375a0ea53fed49)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:18.250183 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4f026-da31-401e-a4c2-541be3b53d64 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:18.251761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 567/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:18 2022] GET /identity/v3/users/b83eed3cde624d0a8f375a0ea53fed49 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:18.257857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41cd42-aaec-449b-bbc8-48461bb9ef71 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:18.352082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:18.378705 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41cd42-aaec-449b-bbc8-48461bb9ef71 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:18.414108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e52f0c-3073-405e-98d1-9e5a05ff19c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:18.414701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e52f0c-3073-405e-98d1-9e5a05ff19c6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:18.415002 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e52f0c-3073-405e-98d1-9e5a05ff19c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:18.415280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e52f0c-3073-405e-98d1-9e5a05ff19c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:18.487104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e52f0c-3073-405e-98d1-9e5a05ff19c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:18.488499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e52f0c-3073-405e-98d1-9e5a05ff19c6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:18.573594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:18.597844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 548/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:18.604377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4f026-da31-401e-a4c2-541be3b53d64 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:18.718710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4f026-da31-401e-a4c2-541be3b53d64 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:18.758164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef84bc-aed0-4f26-a9cf-a3330f350293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:18.758771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef84bc-aed0-4f26-a9cf-a3330f350293 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:18.759027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef84bc-aed0-4f26-a9cf-a3330f350293 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:18.759290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef84bc-aed0-4f26-a9cf-a3330f350293 None admin] PATH_INFO: `/v3/projects/ac563f47bac74778adfa5c58748b654b` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:18.760435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef84bc-aed0-4f26-a9cf-a3330f350293 None admin] RBAC: Authorizing `identity:get_project(project_id=ac563f47bac74778adfa5c58748b654b)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:18.762025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef84bc-aed0-4f26-a9cf-a3330f350293 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:18.763649 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 568/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:18 2022] GET /identity/v3/projects/ac563f47bac74778adfa5c58748b654b => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:18.773839 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e52f0c-3073-405e-98d1-9e5a05ff19c6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:18.885990 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e52f0c-3073-405e-98d1-9e5a05ff19c6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:18.921209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6ed2d-49b2-4598-b6a9-12198d104da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:18.921792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6ed2d-49b2-4598-b6a9-12198d104da8 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:18.922061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6ed2d-49b2-4598-b6a9-12198d104da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:18.922325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6ed2d-49b2-4598-b6a9-12198d104da8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:18.923036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6ed2d-49b2-4598-b6a9-12198d104da8 None admin] RBAC: Adding query filter params (role.id=16317a22e9c94a8886f02efb75ab456e, scope.project.id=ac563f47bac74778adfa5c58748b654b, user.id=b83eed3cde624d0a8f375a0ea53fed49) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:18.923329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6ed2d-49b2-4598-b6a9-12198d104da8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:18.924791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6ed2d-49b2-4598-b6a9-12198d104da8 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:18.930226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 549/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:18 2022] GET /identity/v3/role_assignments?user.id=b83eed3cde624d0a8f375a0ea53fed49&scope.project.id=ac563f47bac74778adfa5c58748b654b&role.id=16317a22e9c94a8886f02efb75ab456e => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:20.790578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace615ba-5c2c-4d4c-b645-9cadcd25e411 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:20.791431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace615ba-5c2c-4d4c-b645-9cadcd25e411 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:20.791688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace615ba-5c2c-4d4c-b645-9cadcd25e411 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:20.793916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 569/1118] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:20.802116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da052c-c237-4a03-8e8e-f6dfabd72314 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:20.802590 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da052c-c237-4a03-8e8e-f6dfabd72314 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:20.802690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da052c-c237-4a03-8e8e-f6dfabd72314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:20.821594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d7da052c-c237-4a03-8e8e-f6dfabd72314 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:20.822782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d7da052c-c237-4a03-8e8e-f6dfabd72314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:20.866707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:20.898290 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 550/1119] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:21:21.185814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace615ba-5c2c-4d4c-b645-9cadcd25e411 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:21.199618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:21.302819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace615ba-5c2c-4d4c-b645-9cadcd25e411 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:21.337469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded082e-7814-4dad-aacf-9a54cd26279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:21.338085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded082e-7814-4dad-aacf-9a54cd26279d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:21.338362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded082e-7814-4dad-aacf-9a54cd26279d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:21.338631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded082e-7814-4dad-aacf-9a54cd26279d None admin] PATH_INFO: `/v3/roles/16317a22e9c94a8886f02efb75ab456e` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:21.340276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded082e-7814-4dad-aacf-9a54cd26279d None admin] RBAC: Authorizing `identity:get_role(role_id=16317a22e9c94a8886f02efb75ab456e)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:21.342035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded082e-7814-4dad-aacf-9a54cd26279d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:21.343404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 570/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:21 2022] GET /identity/v3/roles/16317a22e9c94a8886f02efb75ab456e => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:21.353449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da052c-c237-4a03-8e8e-f6dfabd72314 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:21.480830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da052c-c237-4a03-8e8e-f6dfabd72314 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:21.517250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad77894-3eac-4c4b-9b36-0937e4e1313d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:21.517870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad77894-3eac-4c4b-9b36-0937e4e1313d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:21.518144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad77894-3eac-4c4b-9b36-0937e4e1313d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:21.518408 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad77894-3eac-4c4b-9b36-0937e4e1313d None admin] PATH_INFO: `/v3/users/b83eed3cde624d0a8f375a0ea53fed49` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:21.519569 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad77894-3eac-4c4b-9b36-0937e4e1313d None admin] RBAC: Authorizing `identity:get_user(user_id=b83eed3cde624d0a8f375a0ea53fed49)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:21.521052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad77894-3eac-4c4b-9b36-0937e4e1313d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:21.522641 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 551/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:21 2022] GET /identity/v3/users/b83eed3cde624d0a8f375a0ea53fed49 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:21.531955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded082e-7814-4dad-aacf-9a54cd26279d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:21.622022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:21.653289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded082e-7814-4dad-aacf-9a54cd26279d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:21.689505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ed617-2917-483b-b3ca-8702e34dc065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:21.690122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ed617-2917-483b-b3ca-8702e34dc065 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:21.690395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ed617-2917-483b-b3ca-8702e34dc065 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:21.690660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ed617-2917-483b-b3ca-8702e34dc065 None admin] PATH_INFO: `/v3/projects/ac563f47bac74778adfa5c58748b654b` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:21.691878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ed617-2917-483b-b3ca-8702e34dc065 None admin] RBAC: Authorizing `identity:get_project(project_id=ac563f47bac74778adfa5c58748b654b)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:21.693335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ed617-2917-483b-b3ca-8702e34dc065 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:21.694980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 571/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:21 2022] GET /identity/v3/projects/ac563f47bac74778adfa5c58748b654b => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:21.704815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad77894-3eac-4c4b-9b36-0937e4e1313d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:21.755195 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:21.813168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad77894-3eac-4c4b-9b36-0937e4e1313d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:21.855709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dd577-b526-4cb1-9e4f-f23883e2e518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:21.856785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dd577-b526-4cb1-9e4f-f23883e2e518 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:21.857181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dd577-b526-4cb1-9e4f-f23883e2e518 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:21.857581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dd577-b526-4cb1-9e4f-f23883e2e518 None admin] PATH_INFO: `/v3/projects/ac563f47bac74778adfa5c58748b654b/users/b83eed3cde624d0a8f375a0ea53fed49/roles/16317a22e9c94a8886f02efb75ab456e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:21.861863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dd577-b526-4cb1-9e4f-f23883e2e518 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac563f47bac74778adfa5c58748b654b, user_id=b83eed3cde624d0a8f375a0ea53fed49, role_id=16317a22e9c94a8886f02efb75ab456e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:21.864171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dd577-b526-4cb1-9e4f-f23883e2e518 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:21.883379 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 552/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:21:21 2022] PUT /identity/v3/projects/ac563f47bac74778adfa5c58748b654b/users/b83eed3cde624d0a8f375a0ea53fed49/roles/16317a22e9c94a8886f02efb75ab456e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:21:23.772263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bca22-dd83-41da-9b32-84bbe89c6b5a None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:23.773267 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bca22-dd83-41da-9b32-84bbe89c6b5a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:23.773267 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bca22-dd83-41da-9b32-84bbe89c6b5a None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:23.775439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 572/1124] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:23.783718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02951579-ff2e-4f03-8f6f-6e15a9892c01 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:23.784184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02951579-ff2e-4f03-8f6f-6e15a9892c01 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:23.784281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02951579-ff2e-4f03-8f6f-6e15a9892c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:23.808874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-02951579-ff2e-4f03-8f6f-6e15a9892c01 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:23.810074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-02951579-ff2e-4f03-8f6f-6e15a9892c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:23.884312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 553/1125] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:21:24.149309 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bca22-dd83-41da-9b32-84bbe89c6b5a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:24.276598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97bca22-dd83-41da-9b32-84bbe89c6b5a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:24.321932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a923cc-dca5-49b8-ae01-5733951fa7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:24.322554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a923cc-dca5-49b8-ae01-5733951fa7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:24.322832 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a923cc-dca5-49b8-ae01-5733951fa7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:24.323101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a923cc-dca5-49b8-ae01-5733951fa7b4 None admin] PATH_INFO: `/v3/roles/16317a22e9c94a8886f02efb75ab456e` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:24.324604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a923cc-dca5-49b8-ae01-5733951fa7b4 None admin] RBAC: Authorizing `identity:get_role(role_id=16317a22e9c94a8886f02efb75ab456e)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:24.326021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a923cc-dca5-49b8-ae01-5733951fa7b4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:24.327364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 573/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:24 2022] GET /identity/v3/roles/16317a22e9c94a8886f02efb75ab456e => generated 223 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:24.336934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02951579-ff2e-4f03-8f6f-6e15a9892c01 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:24.371833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:24.442464 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02951579-ff2e-4f03-8f6f-6e15a9892c01 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:24.480763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d9cae-ed22-4b22-b632-2f336c1f6b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:24.481384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d9cae-ed22-4b22-b632-2f336c1f6b60 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:24.481737 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d9cae-ed22-4b22-b632-2f336c1f6b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:24.482021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d9cae-ed22-4b22-b632-2f336c1f6b60 None admin] PATH_INFO: `/v3/users/b83eed3cde624d0a8f375a0ea53fed49` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:24.483273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d9cae-ed22-4b22-b632-2f336c1f6b60 None admin] RBAC: Authorizing `identity:get_user(user_id=b83eed3cde624d0a8f375a0ea53fed49)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:24.484756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d9cae-ed22-4b22-b632-2f336c1f6b60 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:24.486338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 554/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:24 2022] GET /identity/v3/users/b83eed3cde624d0a8f375a0ea53fed49 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:24.493878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a923cc-dca5-49b8-ae01-5733951fa7b4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:24.507363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:24.605304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a923cc-dca5-49b8-ae01-5733951fa7b4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:24.645133 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f8d1f-b71a-4553-894b-9cc9af4e7561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:24.646007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f8d1f-b71a-4553-894b-9cc9af4e7561 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:24.646393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f8d1f-b71a-4553-894b-9cc9af4e7561 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:24.646745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f8d1f-b71a-4553-894b-9cc9af4e7561 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:24.717485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f8d1f-b71a-4553-894b-9cc9af4e7561 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:24.718843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f8d1f-b71a-4553-894b-9cc9af4e7561 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:24.732411 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:24.842343 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 574/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:24.851671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d9cae-ed22-4b22-b632-2f336c1f6b60 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:24.971414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d9cae-ed22-4b22-b632-2f336c1f6b60 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:24.972939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:25.016914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653a16e-196e-4e53-bdcd-acf7f69d68c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:25.017540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653a16e-196e-4e53-bdcd-acf7f69d68c2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:25.017825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653a16e-196e-4e53-bdcd-acf7f69d68c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:25.018093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653a16e-196e-4e53-bdcd-acf7f69d68c2 None admin] PATH_INFO: `/v3/projects/ac563f47bac74778adfa5c58748b654b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:25.019326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653a16e-196e-4e53-bdcd-acf7f69d68c2 None admin] RBAC: Authorizing `identity:get_project(project_id=ac563f47bac74778adfa5c58748b654b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:25.020816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653a16e-196e-4e53-bdcd-acf7f69d68c2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:25.022617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 555/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:24 2022] GET /identity/v3/projects/ac563f47bac74778adfa5c58748b654b => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:25.031641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f8d1f-b71a-4553-894b-9cc9af4e7561 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:25.141742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f8d1f-b71a-4553-894b-9cc9af4e7561 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:25.153209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:25.189775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67115877-a334-4f59-a815-dc219d889728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:25.190370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67115877-a334-4f59-a815-dc219d889728 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:25.190640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67115877-a334-4f59-a815-dc219d889728 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:25.190906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67115877-a334-4f59-a815-dc219d889728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:25.191620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67115877-a334-4f59-a815-dc219d889728 None admin] RBAC: Adding query filter params (role.id=16317a22e9c94a8886f02efb75ab456e, scope.project.id=ac563f47bac74778adfa5c58748b654b, user.id=b83eed3cde624d0a8f375a0ea53fed49) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:25.191920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67115877-a334-4f59-a815-dc219d889728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:25.193382 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67115877-a334-4f59-a815-dc219d889728 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:25.199757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 575/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:25 2022] GET /identity/v3/role_assignments?user.id=b83eed3cde624d0a8f375a0ea53fed49&scope.project.id=ac563f47bac74778adfa5c58748b654b&role.id=16317a22e9c94a8886f02efb75ab456e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:27.146922 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae44992-a2b4-4012-81af-a4259c7eb0ae None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:27.147521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae44992-a2b4-4012-81af-a4259c7eb0ae None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:27.147521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae44992-a2b4-4012-81af-a4259c7eb0ae None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:27.148871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 556/1131] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:27.154576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4fe18-0927-4054-9ae1-9a2f1a43567f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:27.155008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4fe18-0927-4054-9ae1-9a2f1a43567f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:27.155110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4fe18-0927-4054-9ae1-9a2f1a43567f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:27.174758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-83e4fe18-0927-4054-9ae1-9a2f1a43567f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:27.175947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-83e4fe18-0927-4054-9ae1-9a2f1a43567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:27.256279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 576/1132] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:27.523491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae44992-a2b4-4012-81af-a4259c7eb0ae None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:27.648431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae44992-a2b4-4012-81af-a4259c7eb0ae None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:27.685168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f81a-c716-403f-a357-2761ba1bf546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:27.685784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f81a-c716-403f-a357-2761ba1bf546 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:27.686064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f81a-c716-403f-a357-2761ba1bf546 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:27.686334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f81a-c716-403f-a357-2761ba1bf546 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:27.687470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464f81a-c716-403f-a357-2761ba1bf546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:27.688952 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464f81a-c716-403f-a357-2761ba1bf546 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:27.690527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 557/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:27.696907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4fe18-0927-4054-9ae1-9a2f1a43567f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:27.805520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4fe18-0927-4054-9ae1-9a2f1a43567f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:27.807965 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:27.842970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a46c2d-3444-4436-a384-99cbdd4383e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:27.843576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a46c2d-3444-4436-a384-99cbdd4383e3 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:27.843854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a46c2d-3444-4436-a384-99cbdd4383e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:27.844122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a46c2d-3444-4436-a384-99cbdd4383e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:27.844766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a46c2d-3444-4436-a384-99cbdd4383e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:27.846207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a46c2d-3444-4436-a384-99cbdd4383e3 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:27.878859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 577/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:21:27 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:29.815363 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ab2e7-fbf9-4dc8-9f04-eae3570b3218 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:29.815955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ab2e7-fbf9-4dc8-9f04-eae3570b3218 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:29.815955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ab2e7-fbf9-4dc8-9f04-eae3570b3218 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:29.817314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 558/1135] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:29.823668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcb136-799d-4c1f-8c38-5858e5dbb312 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:29.824407 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcb136-799d-4c1f-8c38-5858e5dbb312 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:29.824575 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcb136-799d-4c1f-8c38-5858e5dbb312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:29.849463 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-adbcb136-799d-4c1f-8c38-5858e5dbb312 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:29.850663 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-adbcb136-799d-4c1f-8c38-5858e5dbb312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:29.931173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 578/1136] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:30.203198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ab2e7-fbf9-4dc8-9f04-eae3570b3218 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:30.260531 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:30.323668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ab2e7-fbf9-4dc8-9f04-eae3570b3218 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:30.359077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdeef7-17d2-4009-847e-600443d40069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:30.359698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdeef7-17d2-4009-847e-600443d40069 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:30.359973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdeef7-17d2-4009-847e-600443d40069 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:30.360202 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdeef7-17d2-4009-847e-600443d40069 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:30.361362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdeef7-17d2-4009-847e-600443d40069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:30.362813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdeef7-17d2-4009-847e-600443d40069 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:30.364379 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 559/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:30.371032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcb136-799d-4c1f-8c38-5858e5dbb312 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:30.478536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:30.496140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcb136-799d-4c1f-8c38-5858e5dbb312 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:30.532422 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6087e-0769-4c82-900d-a62824393c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:30.533070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6087e-0769-4c82-900d-a62824393c13 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:30.533348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6087e-0769-4c82-900d-a62824393c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:30.533622 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6087e-0769-4c82-900d-a62824393c13 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:30.534257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6087e-0769-4c82-900d-a62824393c13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:30.535698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6087e-0769-4c82-900d-a62824393c13 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:30.560526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 579/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:21:30 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:32.477551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40e61b-4f00-4275-ac3f-89a564c15acb None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:32.478114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40e61b-4f00-4275-ac3f-89a564c15acb None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:32.478114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40e61b-4f00-4275-ac3f-89a564c15acb None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:32.479460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 560/1139] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:32.486003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d2fd8-1666-468b-b30c-4368bbf81908 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:32.486733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d2fd8-1666-468b-b30c-4368bbf81908 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:32.486920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d2fd8-1666-468b-b30c-4368bbf81908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:32.514601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a04d2fd8-1666-468b-b30c-4368bbf81908 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:32.515809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a04d2fd8-1666-468b-b30c-4368bbf81908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:32.587931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 580/1140] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:32.860574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40e61b-4f00-4275-ac3f-89a564c15acb None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:32.976557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae40e61b-4f00-4275-ac3f-89a564c15acb None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:32.987768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:33.020333 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87256d-4365-4079-9076-c6d012062f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:33.020980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87256d-4365-4079-9076-c6d012062f7f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:33.021258 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87256d-4365-4079-9076-c6d012062f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:33.021527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87256d-4365-4079-9076-c6d012062f7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:33.032276 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87256d-4365-4079-9076-c6d012062f7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-3b87256d-4365-4079-9076-c6d012062f7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:33.037110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:33.038245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:33.039972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:33.039972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:33.039972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:33.039972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:33.039972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:33.039972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:33.039972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:33.039972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:33.039972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:33.039972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:33.039972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:33.039972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:33.043463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 561/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:33.044678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d2fd8-1666-468b-b30c-4368bbf81908 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:33.114239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:33.145902 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d2fd8-1666-468b-b30c-4368bbf81908 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:33.181916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12095e73-96d9-41b2-be2e-f94e83e4fca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:33.182599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12095e73-96d9-41b2-be2e-f94e83e4fca2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:33.182883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12095e73-96d9-41b2-be2e-f94e83e4fca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:33.183148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12095e73-96d9-41b2-be2e-f94e83e4fca2 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:33.183806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12095e73-96d9-41b2-be2e-f94e83e4fca2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:33.184101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12095e73-96d9-41b2-be2e-f94e83e4fca2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:33.185518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12095e73-96d9-41b2-be2e-f94e83e4fca2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:33.193722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 581/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:33.203024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87256d-4365-4079-9076-c6d012062f7f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:33.322848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87256d-4365-4079-9076-c6d012062f7f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:33.359302 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757d0ab-49fe-469a-be20-08f36ef95028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:33.359919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757d0ab-49fe-469a-be20-08f36ef95028 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:33.360199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757d0ab-49fe-469a-be20-08f36ef95028 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:33.360452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757d0ab-49fe-469a-be20-08f36ef95028 None admin] PATH_INFO: `/v3/users/6a5d5e92996c416da151ab52de124777` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:33.374520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757d0ab-49fe-469a-be20-08f36ef95028 None admin] RBAC: Authorizing `identity:get_user(user_id=6a5d5e92996c416da151ab52de124777)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:33.375919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757d0ab-49fe-469a-be20-08f36ef95028 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:33.377586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 562/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:33 2022] GET /identity/v3/users/6a5d5e92996c416da151ab52de124777 => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:33.384573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12095e73-96d9-41b2-be2e-f94e83e4fca2 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:33.495581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12095e73-96d9-41b2-be2e-f94e83e4fca2 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:33.531415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d0c38-bda9-4b5a-8e5f-b2406ccb235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:33.532008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d0c38-bda9-4b5a-8e5f-b2406ccb235e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:33.532279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d0c38-bda9-4b5a-8e5f-b2406ccb235e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:33.532546 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d0c38-bda9-4b5a-8e5f-b2406ccb235e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:33.535383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:33.606870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d0c38-bda9-4b5a-8e5f-b2406ccb235e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:33.609252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d0c38-bda9-4b5a-8e5f-b2406ccb235e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:33.714988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 582/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:33.721461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757d0ab-49fe-469a-be20-08f36ef95028 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:33.747985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:33.829309 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757d0ab-49fe-469a-be20-08f36ef95028 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:33.866974 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0adfa1-d1d5-46e6-903d-f6d89058bbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:33.867711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0adfa1-d1d5-46e6-903d-f6d89058bbfc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:33.868001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0adfa1-d1d5-46e6-903d-f6d89058bbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:33.868272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0adfa1-d1d5-46e6-903d-f6d89058bbfc None admin] PATH_INFO: `/v3/projects/3856153355c34c748864cd5c54c7cb3d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:33.869468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0adfa1-d1d5-46e6-903d-f6d89058bbfc None admin] RBAC: Authorizing `identity:get_project(project_id=3856153355c34c748864cd5c54c7cb3d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:33.870918 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0adfa1-d1d5-46e6-903d-f6d89058bbfc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:33.872566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 563/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:33 2022] GET /identity/v3/projects/3856153355c34c748864cd5c54c7cb3d => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:33.881858 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d0c38-bda9-4b5a-8e5f-b2406ccb235e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:33.918843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:33.994486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d0c38-bda9-4b5a-8e5f-b2406ccb235e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:34.032800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f41f70-910b-4a9b-a204-d0dfe25f9df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:34.033402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f41f70-910b-4a9b-a204-d0dfe25f9df5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:34.033679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f41f70-910b-4a9b-a204-d0dfe25f9df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:34.033952 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f41f70-910b-4a9b-a204-d0dfe25f9df5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:34.034670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f41f70-910b-4a9b-a204-d0dfe25f9df5 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=3856153355c34c748864cd5c54c7cb3d, user.id=6a5d5e92996c416da151ab52de124777) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:34.034979 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f41f70-910b-4a9b-a204-d0dfe25f9df5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:34.036416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f41f70-910b-4a9b-a204-d0dfe25f9df5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:34.041933 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 583/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:33 2022] GET /identity/v3/role_assignments?user.id=6a5d5e92996c416da151ab52de124777&scope.project.id=3856153355c34c748864cd5c54c7cb3d&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:35.910370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f212cf1-747c-4787-acf3-0b6a19534cf0 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:35.911241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f212cf1-747c-4787-acf3-0b6a19534cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:35.911241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f212cf1-747c-4787-acf3-0b6a19534cf0 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:35.913577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 564/1147] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:35.920019 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeae239-ea06-40bd-bdcd-77bd32c1211c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:35.920706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeae239-ea06-40bd-bdcd-77bd32c1211c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:35.920865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeae239-ea06-40bd-bdcd-77bd32c1211c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:35.943259 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-edeae239-ea06-40bd-bdcd-77bd32c1211c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:35.944479 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-edeae239-ea06-40bd-bdcd-77bd32c1211c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:36.022640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 584/1148] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:36.291558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f212cf1-747c-4787-acf3-0b6a19534cf0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:36.401670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:36.413106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f212cf1-747c-4787-acf3-0b6a19534cf0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:36.450833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ef051-25fb-402c-9021-0c6a4f8ee2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:36.451437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ef051-25fb-402c-9021-0c6a4f8ee2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:36.451706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ef051-25fb-402c-9021-0c6a4f8ee2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:36.451970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ef051-25fb-402c-9021-0c6a4f8ee2e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:36.463619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ef051-25fb-402c-9021-0c6a4f8ee2e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-cb3ef051-25fb-402c-9021-0c6a4f8ee2e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:36.468213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:36.472720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:36.473762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 565/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:36.477554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeae239-ea06-40bd-bdcd-77bd32c1211c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:36.511977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:36.593896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeae239-ea06-40bd-bdcd-77bd32c1211c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:36.630468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a0c63-774a-4546-8bf3-bf95ef392a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:36.631069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a0c63-774a-4546-8bf3-bf95ef392a50 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:36.631345 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a0c63-774a-4546-8bf3-bf95ef392a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:36.631617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a0c63-774a-4546-8bf3-bf95ef392a50 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:36.632272 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a0c63-774a-4546-8bf3-bf95ef392a50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:36.632573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a0c63-774a-4546-8bf3-bf95ef392a50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:36.634540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a0c63-774a-4546-8bf3-bf95ef392a50 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:36.642771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 585/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:36.651536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ef051-25fb-402c-9021-0c6a4f8ee2e6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:36.760758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ef051-25fb-402c-9021-0c6a4f8ee2e6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:36.797792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae3d87-321d-43a6-a3bb-6549040475b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:36.798394 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae3d87-321d-43a6-a3bb-6549040475b5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:36.798667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae3d87-321d-43a6-a3bb-6549040475b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:36.798933 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae3d87-321d-43a6-a3bb-6549040475b5 None admin] PATH_INFO: `/v3/users/6a5d5e92996c416da151ab52de124777` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:36.800143 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae3d87-321d-43a6-a3bb-6549040475b5 None admin] RBAC: Authorizing `identity:get_user(user_id=6a5d5e92996c416da151ab52de124777)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:36.801627 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae3d87-321d-43a6-a3bb-6549040475b5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:36.803192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 566/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:36 2022] GET /identity/v3/users/6a5d5e92996c416da151ab52de124777 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:36.813212 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a0c63-774a-4546-8bf3-bf95ef392a50 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:36.916932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:36.927392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a0c63-774a-4546-8bf3-bf95ef392a50 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:36.963206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cba83-1b9b-4650-b8a5-e507eb8594e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:36.963828 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cba83-1b9b-4650-b8a5-e507eb8594e2 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:36.964106 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cba83-1b9b-4650-b8a5-e507eb8594e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:36.964371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cba83-1b9b-4650-b8a5-e507eb8594e2 None admin] PATH_INFO: `/v3/projects/3856153355c34c748864cd5c54c7cb3d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:36.965529 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8cba83-1b9b-4650-b8a5-e507eb8594e2 None admin] RBAC: Authorizing `identity:get_project(project_id=3856153355c34c748864cd5c54c7cb3d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:36.966971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8cba83-1b9b-4650-b8a5-e507eb8594e2 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:36.968618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 586/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:36 2022] GET /identity/v3/projects/3856153355c34c748864cd5c54c7cb3d => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:36.977492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae3d87-321d-43a6-a3bb-6549040475b5 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:37.004062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:37.097620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae3d87-321d-43a6-a3bb-6549040475b5 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:37.144340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467afab8-0963-4a6a-811a-cde793b72b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:37.145057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467afab8-0963-4a6a-811a-cde793b72b93 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:37.145335 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467afab8-0963-4a6a-811a-cde793b72b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:37.145606 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467afab8-0963-4a6a-811a-cde793b72b93 None admin] PATH_INFO: `/v3/projects/3856153355c34c748864cd5c54c7cb3d/users/6a5d5e92996c416da151ab52de124777/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:37.147792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467afab8-0963-4a6a-811a-cde793b72b93 None admin] RBAC: Authorizing `identity:create_grant(project_id=3856153355c34c748864cd5c54c7cb3d, user_id=6a5d5e92996c416da151ab52de124777, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:37.149394 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467afab8-0963-4a6a-811a-cde793b72b93 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:37.170401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 567/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:21:36 2022] PUT /identity/v3/projects/3856153355c34c748864cd5c54c7cb3d/users/6a5d5e92996c416da151ab52de124777/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:21:39.028499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c449f-0b1b-4ce4-acbe-fbe5bbe1a823 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:39.029149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c449f-0b1b-4ce4-acbe-fbe5bbe1a823 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:39.029149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c449f-0b1b-4ce4-acbe-fbe5bbe1a823 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:39.030422 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 587/1154] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:39.036784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fc78b-7330-4e86-9e30-b90d703886b6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:39.037479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fc78b-7330-4e86-9e30-b90d703886b6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:39.037594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fc78b-7330-4e86-9e30-b90d703886b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:39.067441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ac5fc78b-7330-4e86-9e30-b90d703886b6 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:39.069405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ac5fc78b-7330-4e86-9e30-b90d703886b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:39.144031 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 568/1155] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:39.421718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c449f-0b1b-4ce4-acbe-fbe5bbe1a823 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:39.532527 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c449f-0b1b-4ce4-acbe-fbe5bbe1a823 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:39.569765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c1a63-6929-4317-bfe8-0699200ecf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:39.570436 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c1a63-6929-4317-bfe8-0699200ecf21 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:39.570795 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c1a63-6929-4317-bfe8-0699200ecf21 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:39.571068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c1a63-6929-4317-bfe8-0699200ecf21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:39.582449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c1a63-6929-4317-bfe8-0699200ecf21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-278c1a63-6929-4317-bfe8-0699200ecf21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:39.588015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:39.589253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:39.591233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 588/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:39.598973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fc78b-7330-4e86-9e30-b90d703886b6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:39.613556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:39.703566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fc78b-7330-4e86-9e30-b90d703886b6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:39.739980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a090b-205d-42e0-8054-9a27ec119167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:39.740569 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a090b-205d-42e0-8054-9a27ec119167 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:39.740873 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a090b-205d-42e0-8054-9a27ec119167 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:39.741135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a090b-205d-42e0-8054-9a27ec119167 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:39.741794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a090b-205d-42e0-8054-9a27ec119167 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:39.742085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a090b-205d-42e0-8054-9a27ec119167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:39.743491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a090b-205d-42e0-8054-9a27ec119167 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:39.752254 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 569/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:39.758430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c1a63-6929-4317-bfe8-0699200ecf21 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:39.778999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:39.869143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c1a63-6929-4317-bfe8-0699200ecf21 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:39.913233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee089fb3-1284-43b6-a395-511e8a4c6a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:39.914404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee089fb3-1284-43b6-a395-511e8a4c6a58 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:39.914852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee089fb3-1284-43b6-a395-511e8a4c6a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:39.915124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee089fb3-1284-43b6-a395-511e8a4c6a58 None admin] PATH_INFO: `/v3/users/6a5d5e92996c416da151ab52de124777` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:39.916388 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee089fb3-1284-43b6-a395-511e8a4c6a58 None admin] RBAC: Authorizing `identity:get_user(user_id=6a5d5e92996c416da151ab52de124777)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:39.917879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee089fb3-1284-43b6-a395-511e8a4c6a58 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:39.919485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 589/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:39 2022] GET /identity/v3/users/6a5d5e92996c416da151ab52de124777 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:39.926024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a090b-205d-42e0-8054-9a27ec119167 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:40.007245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:40.039187 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a090b-205d-42e0-8054-9a27ec119167 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:40.077206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b66a6a-d289-4729-8b99-d5fab87df0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:40.077803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b66a6a-d289-4729-8b99-d5fab87df0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:40.078080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b66a6a-d289-4729-8b99-d5fab87df0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:40.078345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b66a6a-d289-4729-8b99-d5fab87df0c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:40.149740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b66a6a-d289-4729-8b99-d5fab87df0c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:40.151250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b66a6a-d289-4729-8b99-d5fab87df0c2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:40.230054 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:40.268907 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 570/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:40.275207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee089fb3-1284-43b6-a395-511e8a4c6a58 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:40.386628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:40.405079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee089fb3-1284-43b6-a395-511e8a4c6a58 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:40.442481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c02ef-b30b-4ea6-9c38-7048e21bbdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:40.443102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c02ef-b30b-4ea6-9c38-7048e21bbdf9 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:40.443377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c02ef-b30b-4ea6-9c38-7048e21bbdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:40.443649 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c02ef-b30b-4ea6-9c38-7048e21bbdf9 None admin] PATH_INFO: `/v3/projects/3856153355c34c748864cd5c54c7cb3d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:40.444840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c02ef-b30b-4ea6-9c38-7048e21bbdf9 None admin] RBAC: Authorizing `identity:get_project(project_id=3856153355c34c748864cd5c54c7cb3d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:40.446330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c02ef-b30b-4ea6-9c38-7048e21bbdf9 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:40.447984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 590/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:40 2022] GET /identity/v3/projects/3856153355c34c748864cd5c54c7cb3d => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:40.457276 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b66a6a-d289-4729-8b99-d5fab87df0c2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:40.563939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b66a6a-d289-4729-8b99-d5fab87df0c2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:40.600469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e77b29-49f6-44e0-9e30-a3a9b23921c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:40.601083 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e77b29-49f6-44e0-9e30-a3a9b23921c4 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:40.601403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e77b29-49f6-44e0-9e30-a3a9b23921c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:40.601668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e77b29-49f6-44e0-9e30-a3a9b23921c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:40.602390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e77b29-49f6-44e0-9e30-a3a9b23921c4 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=3856153355c34c748864cd5c54c7cb3d, user.id=6a5d5e92996c416da151ab52de124777) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:40.602691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e77b29-49f6-44e0-9e30-a3a9b23921c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:40.604122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e77b29-49f6-44e0-9e30-a3a9b23921c4 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:40.610026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 571/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:40 2022] GET /identity/v3/role_assignments?user.id=6a5d5e92996c416da151ab52de124777&scope.project.id=3856153355c34c748864cd5c54c7cb3d&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:42.537733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3c498-3da7-4a40-8595-f7f1d90b5756 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:42.538586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3c498-3da7-4a40-8595-f7f1d90b5756 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:42.538586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3c498-3da7-4a40-8595-f7f1d90b5756 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:42.541759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 591/1162] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:42.549560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f0d8e-a477-4ad7-9fd6-63dd7a08ef96 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:42.550228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f0d8e-a477-4ad7-9fd6-63dd7a08ef96 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:42.550494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f0d8e-a477-4ad7-9fd6-63dd7a08ef96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:42.571675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ca5f0d8e-a477-4ad7-9fd6-63dd7a08ef96 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:42.572990 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ca5f0d8e-a477-4ad7-9fd6-63dd7a08ef96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:42.603872 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:42.643417 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 572/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:42.907471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3c498-3da7-4a40-8595-f7f1d90b5756 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:43.030126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3c498-3da7-4a40-8595-f7f1d90b5756 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:43.076828 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fe88e-4c19-42d3-bdbc-4d76f32339b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:43.077420 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fe88e-4c19-42d3-bdbc-4d76f32339b4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:43.077807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fe88e-4c19-42d3-bdbc-4d76f32339b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:43.078082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fe88e-4c19-42d3-bdbc-4d76f32339b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:43.104172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:43.158190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fe88e-4c19-42d3-bdbc-4d76f32339b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:43.159560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fe88e-4c19-42d3-bdbc-4d76f32339b4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:43.270001 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 592/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:43.276449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f0d8e-a477-4ad7-9fd6-63dd7a08ef96 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:43.398522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f0d8e-a477-4ad7-9fd6-63dd7a08ef96 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:43.434547 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbfcc6-a663-4dfd-b74a-8772feac3c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:43.435092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbfcc6-a663-4dfd-b74a-8772feac3c4a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:43.435365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbfcc6-a663-4dfd-b74a-8772feac3c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:43.435630 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbfcc6-a663-4dfd-b74a-8772feac3c4a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:43.442655 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbfcc6-a663-4dfd-b74a-8772feac3c4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:43.444097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbfcc6-a663-4dfd-b74a-8772feac3c4a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-a8bbfcc6-a663-4dfd-b74a-8772feac3c4a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:43.455693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:43.457096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:21:43.458249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 573/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 12:21:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:43.463644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fe88e-4c19-42d3-bdbc-4d76f32339b4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:43.521065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:43.567888 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fe88e-4c19-42d3-bdbc-4d76f32339b4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:43.603757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb7265-0a9d-43d4-beda-4991a6fe3939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:43.604343 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb7265-0a9d-43d4-beda-4991a6fe3939 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:43.604621 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb7265-0a9d-43d4-beda-4991a6fe3939 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:43.604917 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb7265-0a9d-43d4-beda-4991a6fe3939 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:43.605555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb7265-0a9d-43d4-beda-4991a6fe3939 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:43.605859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb7265-0a9d-43d4-beda-4991a6fe3939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:43.607208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb7265-0a9d-43d4-beda-4991a6fe3939 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:43.614434 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 593/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 12:21:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:45.486483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efbae4-99d0-4a96-afdc-3583e958710a None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:45.487087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efbae4-99d0-4a96-afdc-3583e958710a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:45.487087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efbae4-99d0-4a96-afdc-3583e958710a None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:45.488403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 574/1167] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:45.494844 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d679a07-c8a1-4726-8474-b522d4525518 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:45.495585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d679a07-c8a1-4726-8474-b522d4525518 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:45.495733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d679a07-c8a1-4726-8474-b522d4525518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:45.525438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9d679a07-c8a1-4726-8474-b522d4525518 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:45.526624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9d679a07-c8a1-4726-8474-b522d4525518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:45.596614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 594/1168] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:45.866789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efbae4-99d0-4a96-afdc-3583e958710a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:45.898426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:46.006240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efbae4-99d0-4a96-afdc-3583e958710a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:46.047635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc466-c695-4693-8f62-1c9f9b593304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:46.048226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc466-c695-4693-8f62-1c9f9b593304 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:46.048499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc466-c695-4693-8f62-1c9f9b593304 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:46.048813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc466-c695-4693-8f62-1c9f9b593304 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:46.049487 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bc466-c695-4693-8f62-1c9f9b593304 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:46.050868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bc466-c695-4693-8f62-1c9f9b593304 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:46.071016 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 575/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Wed Aug 24 12:21:45 2022] POST /identity/v3/domains => generated 267 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:47.990183 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a7fb3-0615-4ece-a205-a12c6ba0c803 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:47.990954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a7fb3-0615-4ece-a205-a12c6ba0c803 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:47.991055 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a7fb3-0615-4ece-a205-a12c6ba0c803 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:47.993954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 595/1170] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:47.999455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e49f73-0d63-4220-a85d-3bf6f316d770 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:47.999906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e49f73-0d63-4220-a85d-3bf6f316d770 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:48.000007 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e49f73-0d63-4220-a85d-3bf6f316d770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:48.018676 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-89e49f73-0d63-4220-a85d-3bf6f316d770 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:48.019866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-89e49f73-0d63-4220-a85d-3bf6f316d770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:48.111002 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 576/1171] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:48.390033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a7fb3-0615-4ece-a205-a12c6ba0c803 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:48.461502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:48.519591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a7fb3-0615-4ece-a205-a12c6ba0c803 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:48.570050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cd686-2afa-4ced-a6ab-bdfa6f25813f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:48.570673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cd686-2afa-4ced-a6ab-bdfa6f25813f None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:48.570945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cd686-2afa-4ced-a6ab-bdfa6f25813f None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:48.571215 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cd686-2afa-4ced-a6ab-bdfa6f25813f None admin] PATH_INFO: `/v3/domains/0f8e1cf3b3d5487596a61781855a60dc` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:48.582926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cd686-2afa-4ced-a6ab-bdfa6f25813f None admin] RBAC: Authorizing `identity:get_domain(domain_id=0f8e1cf3b3d5487596a61781855a60dc)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:48.584762 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cd686-2afa-4ced-a6ab-bdfa6f25813f None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:48.586730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 596/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Wed Aug 24 12:21:48 2022] GET /identity/v3/domains/0f8e1cf3b3d5487596a61781855a60dc => generated 267 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:48.594333 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e49f73-0d63-4220-a85d-3bf6f316d770 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:48.614334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:48.701307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e49f73-0d63-4220-a85d-3bf6f316d770 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:48.738379 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d701664-a20b-4312-a1e6-767a19975afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:48.738971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d701664-a20b-4312-a1e6-767a19975afe None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:48.739253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d701664-a20b-4312-a1e6-767a19975afe None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:48.739507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d701664-a20b-4312-a1e6-767a19975afe None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:48.740142 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d701664-a20b-4312-a1e6-767a19975afe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:48.741660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d701664-a20b-4312-a1e6-767a19975afe None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:48.770374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 577/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:21:48 2022] POST /identity/v3/projects => generated 361 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:50.758965 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2b95c-6d98-436d-9e78-8a7037277d9f None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:50.759573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2b95c-6d98-436d-9e78-8a7037277d9f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:50.759573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2b95c-6d98-436d-9e78-8a7037277d9f None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:50.760960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 597/1174] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:50.767361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95473bce-8444-449d-8404-2b4212362a36 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:50.768045 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95473bce-8444-449d-8404-2b4212362a36 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:50.768193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95473bce-8444-449d-8404-2b4212362a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:50.790618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-95473bce-8444-449d-8404-2b4212362a36 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:50.791815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-95473bce-8444-449d-8404-2b4212362a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:50.874646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:50.885168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 578/1175] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:51.150720 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2b95c-6d98-436d-9e78-8a7037277d9f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:51.264905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2b95c-6d98-436d-9e78-8a7037277d9f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:51.274314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:51.305622 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d803b2b-afb6-486a-a388-f7866e8679ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:51.306243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d803b2b-afb6-486a-a388-f7866e8679ae None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:51.306523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d803b2b-afb6-486a-a388-f7866e8679ae None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:51.306791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d803b2b-afb6-486a-a388-f7866e8679ae None admin] PATH_INFO: `/v3/domains/0f8e1cf3b3d5487596a61781855a60dc` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:51.308043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d803b2b-afb6-486a-a388-f7866e8679ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=0f8e1cf3b3d5487596a61781855a60dc)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:51.309535 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d803b2b-afb6-486a-a388-f7866e8679ae None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:51.311158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 598/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Wed Aug 24 12:21:51 2022] GET /identity/v3/domains/0f8e1cf3b3d5487596a61781855a60dc => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:51.317234 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95473bce-8444-449d-8404-2b4212362a36 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:51.441762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95473bce-8444-449d-8404-2b4212362a36 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:51.490642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff9846-53f9-4059-a932-8bbf82dd1d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:51.491450 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff9846-53f9-4059-a932-8bbf82dd1d7f None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:51.491792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff9846-53f9-4059-a932-8bbf82dd1d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:51.492025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff9846-53f9-4059-a932-8bbf82dd1d7f None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:51.492687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff9846-53f9-4059-a932-8bbf82dd1d7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:51.494165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff9846-53f9-4059-a932-8bbf82dd1d7f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:51.517030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 579/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:21:51 2022] POST /identity/v3/users => generated 311 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:53.492202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499c816-d1d3-4dce-927d-dc365004d2d5 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:53.493280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499c816-d1d3-4dce-927d-dc365004d2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:53.493280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499c816-d1d3-4dce-927d-dc365004d2d5 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:53.495523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 599/1178] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:53.505137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866ffbe-88cd-4ec4-a5be-d69e5174a04e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:53.505578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866ffbe-88cd-4ec4-a5be-d69e5174a04e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:53.506131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866ffbe-88cd-4ec4-a5be-d69e5174a04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:53.526386 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2866ffbe-88cd-4ec4-a5be-d69e5174a04e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:53.527572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2866ffbe-88cd-4ec4-a5be-d69e5174a04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:53.580577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:53.602645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 580/1179] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:21:53.868394 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499c816-d1d3-4dce-927d-dc365004d2d5 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:53.994403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499c816-d1d3-4dce-927d-dc365004d2d5 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:54.031853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765b354-b9be-440e-a0ad-babbc6bb84f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:54.032485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765b354-b9be-440e-a0ad-babbc6bb84f3 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:54.032792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765b354-b9be-440e-a0ad-babbc6bb84f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:54.033089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765b354-b9be-440e-a0ad-babbc6bb84f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:54.044330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765b354-b9be-440e-a0ad-babbc6bb84f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-5765b354-b9be-440e-a0ad-babbc6bb84f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:54.049060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:54.050192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:54.051049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 600/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:54.058441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866ffbe-88cd-4ec4-a5be-d69e5174a04e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:54.165885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866ffbe-88cd-4ec4-a5be-d69e5174a04e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:54.202364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa634f8-5268-448b-b7af-7c8f53d3571d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:54.202960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa634f8-5268-448b-b7af-7c8f53d3571d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:54.203233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa634f8-5268-448b-b7af-7c8f53d3571d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:54.203502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa634f8-5268-448b-b7af-7c8f53d3571d None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:54.204168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa634f8-5268-448b-b7af-7c8f53d3571d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:54.204463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa634f8-5268-448b-b7af-7c8f53d3571d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:54.205903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa634f8-5268-448b-b7af-7c8f53d3571d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:54.214874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 581/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:54.221689 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765b354-b9be-440e-a0ad-babbc6bb84f3 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:54.262534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:54.331490 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765b354-b9be-440e-a0ad-babbc6bb84f3 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:54.371807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fa289-75b2-470d-aed4-dd3a5da7067a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:54.372429 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fa289-75b2-470d-aed4-dd3a5da7067a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:54.372734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fa289-75b2-470d-aed4-dd3a5da7067a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:54.373012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fa289-75b2-470d-aed4-dd3a5da7067a None admin] PATH_INFO: `/v3/users/21ea879515d54beeacd4174266ef7dfb` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:54.386754 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fa289-75b2-470d-aed4-dd3a5da7067a None admin] RBAC: Authorizing `identity:get_user(user_id=21ea879515d54beeacd4174266ef7dfb)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:54.388221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fa289-75b2-470d-aed4-dd3a5da7067a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:54.389864 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 601/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:54 2022] GET /identity/v3/users/21ea879515d54beeacd4174266ef7dfb => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:54.396025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa634f8-5268-448b-b7af-7c8f53d3571d None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:54.449410 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:54.513757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa634f8-5268-448b-b7af-7c8f53d3571d None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:54.563539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4acf8-eeab-4384-8509-373ef59510f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:54.564134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4acf8-eeab-4384-8509-373ef59510f6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:54.564403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4acf8-eeab-4384-8509-373ef59510f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:54.564694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4acf8-eeab-4384-8509-373ef59510f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:54.636799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4acf8-eeab-4384-8509-373ef59510f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:54.639618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4acf8-eeab-4384-8509-373ef59510f6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:54.683408 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:54.742257 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 582/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:54.748571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fa289-75b2-470d-aed4-dd3a5da7067a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:54.849669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:54.859185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fa289-75b2-470d-aed4-dd3a5da7067a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:54.894824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfaf02-3f26-4075-bbbf-4bd666922922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:54.895459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfaf02-3f26-4075-bbbf-4bd666922922 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:54.895731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfaf02-3f26-4075-bbbf-4bd666922922 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:54.896002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfaf02-3f26-4075-bbbf-4bd666922922 None admin] PATH_INFO: `/v3/projects/241f7b1014854af9bc8bc8bd12e383e4` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:54.897153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfaf02-3f26-4075-bbbf-4bd666922922 None admin] RBAC: Authorizing `identity:get_project(project_id=241f7b1014854af9bc8bc8bd12e383e4)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:54.898768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfaf02-3f26-4075-bbbf-4bd666922922 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:54.900408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 602/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:54 2022] GET /identity/v3/projects/241f7b1014854af9bc8bc8bd12e383e4 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:54.910222 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4acf8-eeab-4384-8509-373ef59510f6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:55.040455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4acf8-eeab-4384-8509-373ef59510f6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:55.081669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5f173-378e-4197-94c4-b62f8d730e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:55.082269 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5f173-378e-4197-94c4-b62f8d730e80 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:55.082548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5f173-378e-4197-94c4-b62f8d730e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:55.082816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5f173-378e-4197-94c4-b62f8d730e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:55.083548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5f173-378e-4197-94c4-b62f8d730e80 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=241f7b1014854af9bc8bc8bd12e383e4, user.id=21ea879515d54beeacd4174266ef7dfb) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:55.084200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5f173-378e-4197-94c4-b62f8d730e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:55.085836 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5f173-378e-4197-94c4-b62f8d730e80 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:55.091930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 583/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:54 2022] GET /identity/v3/role_assignments?user.id=21ea879515d54beeacd4174266ef7dfb&scope.project.id=241f7b1014854af9bc8bc8bd12e383e4&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:56.948953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8abee-945e-4d7a-bd8d-5e862c9fe8d1 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:56.949803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8abee-945e-4d7a-bd8d-5e862c9fe8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:56.949803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8abee-945e-4d7a-bd8d-5e862c9fe8d1 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:56.951974 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 603/1186] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:21:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:56.960366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c8512-b11f-4c31-83e5-a25541126145 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:56.960850 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c8512-b11f-4c31-83e5-a25541126145 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:56.960949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c8512-b11f-4c31-83e5-a25541126145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:56.982832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f33c8512-b11f-4c31-83e5-a25541126145 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:56.984054 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f33c8512-b11f-4c31-83e5-a25541126145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:56.987434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:57.058729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 584/1187] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:21:57.319158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8abee-945e-4d7a-bd8d-5e862c9fe8d1 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:57.427080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8abee-945e-4d7a-bd8d-5e862c9fe8d1 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:57.463031 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02259f10-1b0d-4d9e-9f1d-515f156109be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:57.463637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02259f10-1b0d-4d9e-9f1d-515f156109be None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:57.463911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02259f10-1b0d-4d9e-9f1d-515f156109be None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:57.464175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02259f10-1b0d-4d9e-9f1d-515f156109be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:57.474210 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02259f10-1b0d-4d9e-9f1d-515f156109be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-02259f10-1b0d-4d9e-9f1d-515f156109be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:57.479129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:57.480363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:21:57.481234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 604/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:57.487066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c8512-b11f-4c31-83e5-a25541126145 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:57.604475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c8512-b11f-4c31-83e5-a25541126145 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:57.644864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c846353-6c49-48ca-97c3-5e28c7cc3c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:57.645467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c846353-6c49-48ca-97c3-5e28c7cc3c8b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:57.645743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c846353-6c49-48ca-97c3-5e28c7cc3c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:57.646013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c846353-6c49-48ca-97c3-5e28c7cc3c8b None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:57.646680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c846353-6c49-48ca-97c3-5e28c7cc3c8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:57.646987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c846353-6c49-48ca-97c3-5e28c7cc3c8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:57.649431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c846353-6c49-48ca-97c3-5e28c7cc3c8b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:57.659663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 585/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:57.667021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02259f10-1b0d-4d9e-9f1d-515f156109be None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:57.702444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:57.790924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02259f10-1b0d-4d9e-9f1d-515f156109be None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:57.828583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943a8cf-1e85-4dfe-ac62-104b0ea4d31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:57.829229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943a8cf-1e85-4dfe-ac62-104b0ea4d31b None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:57.829503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943a8cf-1e85-4dfe-ac62-104b0ea4d31b None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:57.829769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943a8cf-1e85-4dfe-ac62-104b0ea4d31b None admin] PATH_INFO: `/v3/users/21ea879515d54beeacd4174266ef7dfb` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:57.830860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943a8cf-1e85-4dfe-ac62-104b0ea4d31b None admin] RBAC: Authorizing `identity:get_user(user_id=21ea879515d54beeacd4174266ef7dfb)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:57.832310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943a8cf-1e85-4dfe-ac62-104b0ea4d31b None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:57.834298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 605/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:57 2022] GET /identity/v3/users/21ea879515d54beeacd4174266ef7dfb => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:57.841197 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c846353-6c49-48ca-97c3-5e28c7cc3c8b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:57.845775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:57.949717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c846353-6c49-48ca-97c3-5e28c7cc3c8b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:57.987196 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e94591-b605-4c10-b79f-32863ed3136e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:57.987776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e94591-b605-4c10-b79f-32863ed3136e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:57.988083 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e94591-b605-4c10-b79f-32863ed3136e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:57.988351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e94591-b605-4c10-b79f-32863ed3136e None admin] PATH_INFO: `/v3/projects/241f7b1014854af9bc8bc8bd12e383e4` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:57.989634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e94591-b605-4c10-b79f-32863ed3136e None admin] RBAC: Authorizing `identity:get_project(project_id=241f7b1014854af9bc8bc8bd12e383e4)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:57.991087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e94591-b605-4c10-b79f-32863ed3136e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:57.992936 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 586/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:57 2022] GET /identity/v3/projects/241f7b1014854af9bc8bc8bd12e383e4 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:58.002995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943a8cf-1e85-4dfe-ac62-104b0ea4d31b None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:58.108448 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:58.120917 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943a8cf-1e85-4dfe-ac62-104b0ea4d31b None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:58.156435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d334bfe-7942-4f1e-a98f-7c296436ba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:58.157114 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d334bfe-7942-4f1e-a98f-7c296436ba25 None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:58.157390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d334bfe-7942-4f1e-a98f-7c296436ba25 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:58.157654 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d334bfe-7942-4f1e-a98f-7c296436ba25 None admin] PATH_INFO: `/v3/projects/241f7b1014854af9bc8bc8bd12e383e4/users/21ea879515d54beeacd4174266ef7dfb/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:58.159653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d334bfe-7942-4f1e-a98f-7c296436ba25 None admin] RBAC: Authorizing `identity:create_grant(project_id=241f7b1014854af9bc8bc8bd12e383e4, user_id=21ea879515d54beeacd4174266ef7dfb, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:58.161134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d334bfe-7942-4f1e-a98f-7c296436ba25 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:58.180119 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 606/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:21:58 2022] PUT /identity/v3/projects/241f7b1014854af9bc8bc8bd12e383e4/users/21ea879515d54beeacd4174266ef7dfb/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:22:00.113569 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbfaf0-6b1c-4a79-8123-cc5dff72c660 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:00.114458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbfaf0-6b1c-4a79-8123-cc5dff72c660 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:00.114458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbfaf0-6b1c-4a79-8123-cc5dff72c660 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:00.116885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 587/1193] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:22:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:00.126126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005c293-4ba0-4c99-8694-9ea0f5e7b448 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:00.126884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005c293-4ba0-4c99-8694-9ea0f5e7b448 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:00.127016 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005c293-4ba0-4c99-8694-9ea0f5e7b448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:00.149566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8005c293-4ba0-4c99-8694-9ea0f5e7b448 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:00.150742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8005c293-4ba0-4c99-8694-9ea0f5e7b448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:00.227445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 607/1194] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:22:00.506119 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbfaf0-6b1c-4a79-8123-cc5dff72c660 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:00.591545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:00.633343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbfaf0-6b1c-4a79-8123-cc5dff72c660 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:00.667086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb390d2-2f46-4017-ad84-b3aa3db942fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:00.667697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb390d2-2f46-4017-ad84-b3aa3db942fb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:00.667967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb390d2-2f46-4017-ad84-b3aa3db942fb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:00.668230 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb390d2-2f46-4017-ad84-b3aa3db942fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:00.679198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb390d2-2f46-4017-ad84-b3aa3db942fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-6cb390d2-2f46-4017-ad84-b3aa3db942fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:22:00.683764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:00.684970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:22:00.686239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 588/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:22:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:22:00.691272 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005c293-4ba0-4c99-8694-9ea0f5e7b448 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:00.774668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:00.800277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005c293-4ba0-4c99-8694-9ea0f5e7b448 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:00.835928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013f876-6102-4c74-ae84-1b8d96bb2859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:00.836514 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013f876-6102-4c74-ae84-1b8d96bb2859 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:00.836812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013f876-6102-4c74-ae84-1b8d96bb2859 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:00.837190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013f876-6102-4c74-ae84-1b8d96bb2859 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:00.837842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013f876-6102-4c74-ae84-1b8d96bb2859 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:00.838140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013f876-6102-4c74-ae84-1b8d96bb2859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:00.839508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013f876-6102-4c74-ae84-1b8d96bb2859 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:00.847573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 608/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:22:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:22:00.854153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb390d2-2f46-4017-ad84-b3aa3db942fb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:00.985172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb390d2-2f46-4017-ad84-b3aa3db942fb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:01.024069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:01.030852 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa1b72-7e30-47cb-ab42-082413663e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:01.031428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa1b72-7e30-47cb-ab42-082413663e57 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:01.031711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa1b72-7e30-47cb-ab42-082413663e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:01.031988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa1b72-7e30-47cb-ab42-082413663e57 None admin] PATH_INFO: `/v3/users/21ea879515d54beeacd4174266ef7dfb` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:01.033196 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa1b72-7e30-47cb-ab42-082413663e57 None admin] RBAC: Authorizing `identity:get_user(user_id=21ea879515d54beeacd4174266ef7dfb)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:01.034654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa1b72-7e30-47cb-ab42-082413663e57 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:01.036287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 589/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 12:22:00 2022] GET /identity/v3/users/21ea879515d54beeacd4174266ef7dfb => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:22:01.042845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013f876-6102-4c74-ae84-1b8d96bb2859 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:01.146756 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013f876-6102-4c74-ae84-1b8d96bb2859 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:01.185331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c84f6-01e8-4011-81fc-3bec6fc7bc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:01.185856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c84f6-01e8-4011-81fc-3bec6fc7bc71 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:01.186125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c84f6-01e8-4011-81fc-3bec6fc7bc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:01.186393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c84f6-01e8-4011-81fc-3bec6fc7bc71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:01.198520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:01.273120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c84f6-01e8-4011-81fc-3bec6fc7bc71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:01.274563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c84f6-01e8-4011-81fc-3bec6fc7bc71 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:01.383279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 609/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:22:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:22:01.393191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa1b72-7e30-47cb-ab42-082413663e57 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:01.506849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa1b72-7e30-47cb-ab42-082413663e57 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:01.543479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36890a06-93d5-4c89-b5a2-bb0527e07067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:01.544097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36890a06-93d5-4c89-b5a2-bb0527e07067 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:01.544375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36890a06-93d5-4c89-b5a2-bb0527e07067 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:01.544680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36890a06-93d5-4c89-b5a2-bb0527e07067 None admin] PATH_INFO: `/v3/projects/241f7b1014854af9bc8bc8bd12e383e4` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:01.545934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36890a06-93d5-4c89-b5a2-bb0527e07067 None admin] RBAC: Authorizing `identity:get_project(project_id=241f7b1014854af9bc8bc8bd12e383e4)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:01.547396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36890a06-93d5-4c89-b5a2-bb0527e07067 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:01.549173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 590/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 12:22:01 2022] GET /identity/v3/projects/241f7b1014854af9bc8bc8bd12e383e4 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:22:01.559356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c84f6-01e8-4011-81fc-3bec6fc7bc71 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:01.607985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:01.668327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c84f6-01e8-4011-81fc-3bec6fc7bc71 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:01.706690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44ea41-d9cb-41e7-905b-792b068bf1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:01.707280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44ea41-d9cb-41e7-905b-792b068bf1ff None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:01.707555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44ea41-d9cb-41e7-905b-792b068bf1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:01.707820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44ea41-d9cb-41e7-905b-792b068bf1ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:01.708917 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44ea41-d9cb-41e7-905b-792b068bf1ff None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=241f7b1014854af9bc8bc8bd12e383e4, user.id=21ea879515d54beeacd4174266ef7dfb) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:01.709222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44ea41-d9cb-41e7-905b-792b068bf1ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:01.710869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44ea41-d9cb-41e7-905b-792b068bf1ff None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:01.717286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 610/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:22:01 2022] GET /identity/v3/role_assignments?user.id=21ea879515d54beeacd4174266ef7dfb&scope.project.id=241f7b1014854af9bc8bc8bd12e383e4&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:22:11.538175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=67 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:11.714551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=47 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:58.256899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c96b99-daee-4720-b0dc-b73d34b573a1 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:58.257495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c96b99-daee-4720-b0dc-b73d34b573a1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:58.257495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c96b99-daee-4720-b0dc-b73d34b573a1 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:58.259627 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 591/1201] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:22:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:58.265910 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03170e22-1a84-456d-ba0d-be5a4f6a6d2a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:58.266419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03170e22-1a84-456d-ba0d-be5a4f6a6d2a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:58.266835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03170e22-1a84-456d-ba0d-be5a4f6a6d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:58.294126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-03170e22-1a84-456d-ba0d-be5a4f6a6d2a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:58.295754 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-03170e22-1a84-456d-ba0d-be5a4f6a6d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:58.393058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 611/1202] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:22:58.664800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c96b99-daee-4720-b0dc-b73d34b573a1 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:58.800118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c96b99-daee-4720-b0dc-b73d34b573a1 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:58.854212 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304ba43-e311-4816-a9ca-27863a5dd361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:58.854986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304ba43-e311-4816-a9ca-27863a5dd361 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:58.855354 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304ba43-e311-4816-a9ca-27863a5dd361 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:58.855675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304ba43-e311-4816-a9ca-27863a5dd361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:58.870671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304ba43-e311-4816-a9ca-27863a5dd361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:58.872599 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304ba43-e311-4816-a9ca-27863a5dd361 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-1304ba43-e311-4816-a9ca-27863a5dd361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:22:58.896691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:58.897859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:22:58.898820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:22:58.898820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:22:58.898820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:22:58.898820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 592/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:22:58.904493 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03170e22-1a84-456d-ba0d-be5a4f6a6d2a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:59.028309 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03170e22-1a84-456d-ba0d-be5a4f6a6d2a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:59.071559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:59.079413 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d56b98-13ee-45ad-b14c-de33e0160bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:59.080880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d56b98-13ee-45ad-b14c-de33e0160bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:59.081243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d56b98-13ee-45ad-b14c-de33e0160bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:59.081560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d56b98-13ee-45ad-b14c-de33e0160bd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:59.082711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d56b98-13ee-45ad-b14c-de33e0160bd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:59.083081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d56b98-13ee-45ad-b14c-de33e0160bd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:59.084907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d56b98-13ee-45ad-b14c-de33e0160bd1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:59.100965 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 612/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:22:59.108572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304ba43-e311-4816-a9ca-27863a5dd361 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:59.189734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:59.250445 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304ba43-e311-4816-a9ca-27863a5dd361 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:59.287631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce068ab0-c18b-45e4-a121-bbd35889c6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:59.288426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce068ab0-c18b-45e4-a121-bbd35889c6c9 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:59.288426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce068ab0-c18b-45e4-a121-bbd35889c6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:59.288753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce068ab0-c18b-45e4-a121-bbd35889c6c9 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:59.289345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce068ab0-c18b-45e4-a121-bbd35889c6c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:59.290791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce068ab0-c18b-45e4-a121-bbd35889c6c9 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:59.323279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 593/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 12:22:59 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:23:01.377102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a76ba-a6d7-491e-8ff7-373e73c49b43 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:01.377667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a76ba-a6d7-491e-8ff7-373e73c49b43 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:01.377667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a76ba-a6d7-491e-8ff7-373e73c49b43 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:01.379752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 613/1206] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:23:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:01.386219 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f1e28-06d9-48c8-a1d4-4da54fca7f0f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:01.386787 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f1e28-06d9-48c8-a1d4-4da54fca7f0f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:01.386881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f1e28-06d9-48c8-a1d4-4da54fca7f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:01.406766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e74f1e28-06d9-48c8-a1d4-4da54fca7f0f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:01.408043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e74f1e28-06d9-48c8-a1d4-4da54fca7f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:01.491656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 594/1207] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:01.800033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a76ba-a6d7-491e-8ff7-373e73c49b43 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:01.937360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a76ba-a6d7-491e-8ff7-373e73c49b43 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:01.986985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649ffc7-7865-4671-bfbb-039f488586bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:01.987829 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649ffc7-7865-4671-bfbb-039f488586bd None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:01.988323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649ffc7-7865-4671-bfbb-039f488586bd None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:01.988757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649ffc7-7865-4671-bfbb-039f488586bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.000435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649ffc7-7865-4671-bfbb-039f488586bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-4649ffc7-7865-4671-bfbb-039f488586bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:23:02.006361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:02.007605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:02.008593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 614/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:23:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:02.014207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f1e28-06d9-48c8-a1d4-4da54fca7f0f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:02.064441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:02.148093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f1e28-06d9-48c8-a1d4-4da54fca7f0f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:02.193544 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6a38b-0963-4355-825d-6a0c6f1864c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:02.194167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6a38b-0963-4355-825d-6a0c6f1864c1 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:02.194442 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6a38b-0963-4355-825d-6a0c6f1864c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:02.194710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6a38b-0963-4355-825d-6a0c6f1864c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.195386 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6a38b-0963-4355-825d-6a0c6f1864c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:02.195740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6a38b-0963-4355-825d-6a0c6f1864c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:02.197119 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6a38b-0963-4355-825d-6a0c6f1864c1 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:02.206686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 595/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:23:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:23:02.212822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649ffc7-7865-4671-bfbb-039f488586bd None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:02.275016 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:02.336144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649ffc7-7865-4671-bfbb-039f488586bd None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:02.384076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231c8e8-da50-41cf-8434-44453401e9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:02.384852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231c8e8-da50-41cf-8434-44453401e9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:02.385043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231c8e8-da50-41cf-8434-44453401e9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:02.385339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231c8e8-da50-41cf-8434-44453401e9c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.397338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231c8e8-da50-41cf-8434-44453401e9c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:02.401249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231c8e8-da50-41cf-8434-44453401e9c5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-a231c8e8-da50-41cf-8434-44453401e9c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:02.419353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:02.425505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:02.426671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:02.426671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:02.426671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:02.426671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 615/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:02.426918 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6a38b-0963-4355-825d-6a0c6f1864c1 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:02.554677 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:02.558692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6a38b-0963-4355-825d-6a0c6f1864c1 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:02.602655 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ec1a6-b59d-4b0f-a73f-0232b0a5a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:02.603508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ec1a6-b59d-4b0f-a73f-0232b0a5a8ca None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:02.603898 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ec1a6-b59d-4b0f-a73f-0232b0a5a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:02.604291 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ec1a6-b59d-4b0f-a73f-0232b0a5a8ca None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.605002 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ec1a6-b59d-4b0f-a73f-0232b0a5a8ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:02.605292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ec1a6-b59d-4b0f-a73f-0232b0a5a8ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:02.606751 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ec1a6-b59d-4b0f-a73f-0232b0a5a8ca None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:02.621542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 596/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:02.627498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231c8e8-da50-41cf-8434-44453401e9c5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:02.746807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231c8e8-da50-41cf-8434-44453401e9c5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:02.748709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:02.786566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698e31d-333c-496a-9042-9cfdab81de16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:02.787405 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698e31d-333c-496a-9042-9cfdab81de16 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:02.787769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698e31d-333c-496a-9042-9cfdab81de16 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:02.788105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698e31d-333c-496a-9042-9cfdab81de16 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.796845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698e31d-333c-496a-9042-9cfdab81de16 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:02.798394 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698e31d-333c-496a-9042-9cfdab81de16 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6698e31d-333c-496a-9042-9cfdab81de16 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:23:02.811665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:02.818794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:02.819995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:02.819995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:02.819995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:23:02.819995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:23:02.819995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:23:02.819995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:23:02.819995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:23:02.819995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:23:02.819995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:02.819995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:02.819995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 616/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:23:02.822133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ec1a6-b59d-4b0f-a73f-0232b0a5a8ca None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:02.943273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ec1a6-b59d-4b0f-a73f-0232b0a5a8ca None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:02.984539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ce495-1472-444c-b29b-3982472c4cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:02.985765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ce495-1472-444c-b29b-3982472c4cdb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:02.985765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ce495-1472-444c-b29b-3982472c4cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:02.985765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ce495-1472-444c-b29b-3982472c4cdb None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.986439 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ce495-1472-444c-b29b-3982472c4cdb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:02.986754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ce495-1472-444c-b29b-3982472c4cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:02.988309 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ce495-1472-444c-b29b-3982472c4cdb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:03.010366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 597/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 24 12:23:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:03.016220 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698e31d-333c-496a-9042-9cfdab81de16 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:03.144990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698e31d-333c-496a-9042-9cfdab81de16 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:03.185716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe24f73-ccdb-497a-a66a-739cf2d83676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:03.186331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe24f73-ccdb-497a-a66a-739cf2d83676 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:03.186606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe24f73-ccdb-497a-a66a-739cf2d83676 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:03.186865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe24f73-ccdb-497a-a66a-739cf2d83676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:03.224955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:03.272597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe24f73-ccdb-497a-a66a-739cf2d83676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:03.274093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe24f73-ccdb-497a-a66a-739cf2d83676 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:03.418100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 617/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:23:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:23:03.423411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ce495-1472-444c-b29b-3982472c4cdb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:03.449680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:03.544478 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ce495-1472-444c-b29b-3982472c4cdb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:03.585020 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1bb72-971e-45a8-be76-31d3867e660b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:03.585671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1bb72-971e-45a8-be76-31d3867e660b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:03.585941 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1bb72-971e-45a8-be76-31d3867e660b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:03.586205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1bb72-971e-45a8-be76-31d3867e660b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:03.601554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1bb72-971e-45a8-be76-31d3867e660b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:03.604186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1bb72-971e-45a8-be76-31d3867e660b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-c2e1bb72-971e-45a8-be76-31d3867e660b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:03.625366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:03.626422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:03.627373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:03.627373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:03.627373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:03.627373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 598/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:23:03.633928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe24f73-ccdb-497a-a66a-739cf2d83676 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:03.713698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:03.762963 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe24f73-ccdb-497a-a66a-739cf2d83676 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:03.806394 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683c338-eff2-4be6-92ce-6f5efef53d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:03.807531 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683c338-eff2-4be6-92ce-6f5efef53d9c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:03.808223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683c338-eff2-4be6-92ce-6f5efef53d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:03.808840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683c338-eff2-4be6-92ce-6f5efef53d9c None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:03.809885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683c338-eff2-4be6-92ce-6f5efef53d9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:03.810349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683c338-eff2-4be6-92ce-6f5efef53d9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:03.812088 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683c338-eff2-4be6-92ce-6f5efef53d9c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:03.827628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 618/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:03.833962 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1bb72-971e-45a8-be76-31d3867e660b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:03.916462 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:03.964594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1bb72-971e-45a8-be76-31d3867e660b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:04.011481 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d555ba-22cf-4d3e-8329-fcb4d94b7c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:04.012228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d555ba-22cf-4d3e-8329-fcb4d94b7c2b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:04.012517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d555ba-22cf-4d3e-8329-fcb4d94b7c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:04.012828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d555ba-22cf-4d3e-8329-fcb4d94b7c2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:04.020574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d555ba-22cf-4d3e-8329-fcb4d94b7c2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:04.022081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d555ba-22cf-4d3e-8329-fcb4d94b7c2b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-48d555ba-22cf-4d3e-8329-fcb4d94b7c2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:23:04.034466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:04.035722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:04.041753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:23:04.041753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:23:04.041753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:23:04.041753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:23:04.041753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:23:04.041753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:23:04.041753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:04.041753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:04.041753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 599/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:23:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:04.042401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683c338-eff2-4be6-92ce-6f5efef53d9c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:04.163919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683c338-eff2-4be6-92ce-6f5efef53d9c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:04.189433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:04.206216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f1613-8225-46b1-aeb9-f1cd5ae84aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:04.207176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f1613-8225-46b1-aeb9-f1cd5ae84aff None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:04.207521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f1613-8225-46b1-aeb9-f1cd5ae84aff None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:04.207909 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f1613-8225-46b1-aeb9-f1cd5ae84aff None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:04.208790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f1613-8225-46b1-aeb9-f1cd5ae84aff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:04.209208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f1613-8225-46b1-aeb9-f1cd5ae84aff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:04.211026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f1613-8225-46b1-aeb9-f1cd5ae84aff None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:04.229921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 619/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:23:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:23:04.231282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d555ba-22cf-4d3e-8329-fcb4d94b7c2b None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:04.355965 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d555ba-22cf-4d3e-8329-fcb4d94b7c2b None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:04.391020 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:04.397607 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a090c-cbaa-48ab-8c65-b130b4780a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:04.398233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a090c-cbaa-48ab-8c65-b130b4780a2d None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:04.398480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a090c-cbaa-48ab-8c65-b130b4780a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:04.398791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a090c-cbaa-48ab-8c65-b130b4780a2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:04.399542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a090c-cbaa-48ab-8c65-b130b4780a2d None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=6db648aaa06b476e9a3cafbab832ee68) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:04.399888 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a090c-cbaa-48ab-8c65-b130b4780a2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:04.401460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a090c-cbaa-48ab-8c65-b130b4780a2d None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:04.407246 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 600/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:23:04 2022] GET /identity/v3/role_assignments?user.id=6db648aaa06b476e9a3cafbab832ee68&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:06.319228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08496824-0052-4bec-bcdd-88324001210a None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:06.319772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08496824-0052-4bec-bcdd-88324001210a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:06.319772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08496824-0052-4bec-bcdd-88324001210a None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:06.321279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 620/1220] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:06.326837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3503582-2f2f-42c4-84ac-7e715c38aa55 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:06.327272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3503582-2f2f-42c4-84ac-7e715c38aa55 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:06.327418 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3503582-2f2f-42c4-84ac-7e715c38aa55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:06.349051 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e3503582-2f2f-42c4-84ac-7e715c38aa55 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:06.350338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e3503582-2f2f-42c4-84ac-7e715c38aa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:06.450877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 601/1221] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:06.747696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08496824-0052-4bec-bcdd-88324001210a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:06.862102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08496824-0052-4bec-bcdd-88324001210a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:06.905395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f524e-9fbf-47a6-9970-27f01277f764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:06.906095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f524e-9fbf-47a6-9970-27f01277f764 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:06.906408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f524e-9fbf-47a6-9970-27f01277f764 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:06.906874 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f524e-9fbf-47a6-9970-27f01277f764 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:06.918183 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f524e-9fbf-47a6-9970-27f01277f764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-101f524e-9fbf-47a6-9970-27f01277f764 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:23:06.922904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:06.924052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:06.925189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 621/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:23:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:06.930621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3503582-2f2f-42c4-84ac-7e715c38aa55 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:07.053586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3503582-2f2f-42c4-84ac-7e715c38aa55 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:07.077487 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:07.089799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59d869-c602-4b54-b34c-dc3bbf4cb4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:07.090414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59d869-c602-4b54-b34c-dc3bbf4cb4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:07.090684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59d869-c602-4b54-b34c-dc3bbf4cb4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:07.090937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59d869-c602-4b54-b34c-dc3bbf4cb4f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:07.091614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59d869-c602-4b54-b34c-dc3bbf4cb4f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:07.091893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59d869-c602-4b54-b34c-dc3bbf4cb4f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:07.093308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59d869-c602-4b54-b34c-dc3bbf4cb4f5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:07.101423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 602/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:23:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:07.108082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f524e-9fbf-47a6-9970-27f01277f764 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:07.156566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:07.216510 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f524e-9fbf-47a6-9970-27f01277f764 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:07.255395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e922c-948b-4560-acb0-1609203c0ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:07.256060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e922c-948b-4560-acb0-1609203c0ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:07.256339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e922c-948b-4560-acb0-1609203c0ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:07.256613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e922c-948b-4560-acb0-1609203c0ce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:07.267069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e922c-948b-4560-acb0-1609203c0ce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:07.268694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e922c-948b-4560-acb0-1609203c0ce6 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6c7e922c-948b-4560-acb0-1609203c0ce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:07.285709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:07.286928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:07.288039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:07.288039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:07.288039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:07.288039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 622/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:07.293124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59d869-c602-4b54-b34c-dc3bbf4cb4f5 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:07.418625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59d869-c602-4b54-b34c-dc3bbf4cb4f5 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:07.458312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28222324-6661-431a-aff3-ca73b04bdecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:07.458915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28222324-6661-431a-aff3-ca73b04bdecb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:07.459187 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28222324-6661-431a-aff3-ca73b04bdecb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:07.459450 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28222324-6661-431a-aff3-ca73b04bdecb None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:07.460089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28222324-6661-431a-aff3-ca73b04bdecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:07.460389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28222324-6661-431a-aff3-ca73b04bdecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:07.461840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28222324-6661-431a-aff3-ca73b04bdecb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:07.474505 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 603/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:07.480578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e922c-948b-4560-acb0-1609203c0ce6 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:07.588270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:07.595667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e922c-948b-4560-acb0-1609203c0ce6 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:07.628777 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813dc6e-b36e-4d8b-afcd-6e3d85317334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:07.629645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813dc6e-b36e-4d8b-afcd-6e3d85317334 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:07.629920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813dc6e-b36e-4d8b-afcd-6e3d85317334 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:07.630185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813dc6e-b36e-4d8b-afcd-6e3d85317334 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:07.637763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813dc6e-b36e-4d8b-afcd-6e3d85317334 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:07.639208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813dc6e-b36e-4d8b-afcd-6e3d85317334 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-2813dc6e-b36e-4d8b-afcd-6e3d85317334 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:23:07.650435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:07.651603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:07.652781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:07.652781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:07.652781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:23:07.652781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:23:07.652781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:23:07.652781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:23:07.652781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:23:07.652781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:23:07.652781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:07.652781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:07.652781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 623/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:07.657711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28222324-6661-431a-aff3-ca73b04bdecb None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:07.713288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:07.780929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28222324-6661-431a-aff3-ca73b04bdecb None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:07.819092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c54c5-070b-4de9-9628-02d88f0b9c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:07.819700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c54c5-070b-4de9-9628-02d88f0b9c18 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:07.819974 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c54c5-070b-4de9-9628-02d88f0b9c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:07.820242 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c54c5-070b-4de9-9628-02d88f0b9c18 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:07.820999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c54c5-070b-4de9-9628-02d88f0b9c18 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:07.821300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c54c5-070b-4de9-9628-02d88f0b9c18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:07.822746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c54c5-070b-4de9-9628-02d88f0b9c18 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:07.841365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 604/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 24 12:23:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:07.847417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813dc6e-b36e-4d8b-afcd-6e3d85317334 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:07.974221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813dc6e-b36e-4d8b-afcd-6e3d85317334 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:08.015181 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e89a2-7aa8-4f4a-b73e-d7b1a1a97231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:08.015876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e89a2-7aa8-4f4a-b73e-d7b1a1a97231 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:08.016155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e89a2-7aa8-4f4a-b73e-d7b1a1a97231 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:08.016426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e89a2-7aa8-4f4a-b73e-d7b1a1a97231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:08.027501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e89a2-7aa8-4f4a-b73e-d7b1a1a97231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:08.029110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e89a2-7aa8-4f4a-b73e-d7b1a1a97231 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3d8e89a2-7aa8-4f4a-b73e-d7b1a1a97231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:08.046349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:08.047438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:08.048404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:08.048404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:08.048404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:08.048404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 624/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:23:08.054253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c54c5-070b-4de9-9628-02d88f0b9c18 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:08.197230 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c54c5-070b-4de9-9628-02d88f0b9c18 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:08.199924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:08.243292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba4593-4ac4-42d6-bb56-da17a99c66c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:08.243822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba4593-4ac4-42d6-bb56-da17a99c66c2 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:08.244115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba4593-4ac4-42d6-bb56-da17a99c66c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:08.244373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba4593-4ac4-42d6-bb56-da17a99c66c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:08.245088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba4593-4ac4-42d6-bb56-da17a99c66c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:08.245388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba4593-4ac4-42d6-bb56-da17a99c66c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:08.246859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ba4593-4ac4-42d6-bb56-da17a99c66c2 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:08.261762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 605/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:08.267813 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e89a2-7aa8-4f4a-b73e-d7b1a1a97231 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:08.278147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:08.406098 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e89a2-7aa8-4f4a-b73e-d7b1a1a97231 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:08.447279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd916964-47e9-4882-9029-1a522dd5305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:08.447940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd916964-47e9-4882-9029-1a522dd5305e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:08.448261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd916964-47e9-4882-9029-1a522dd5305e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:08.449588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd916964-47e9-4882-9029-1a522dd5305e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:08.461535 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd916964-47e9-4882-9029-1a522dd5305e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:08.463586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd916964-47e9-4882-9029-1a522dd5305e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-dd916964-47e9-4882-9029-1a522dd5305e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:23:08.481123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:08.483316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:08.484789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:23:08.484789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:23:08.484789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:23:08.484789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:23:08.484789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:23:08.484789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:23:08.484789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:08.484789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:08.484789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 625/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:23:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:08.491857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba4593-4ac4-42d6-bb56-da17a99c66c2 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:08.632669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba4593-4ac4-42d6-bb56-da17a99c66c2 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:08.673436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e048f51-3e70-4f23-8333-fe0c11f312e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:08.674164 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e048f51-3e70-4f23-8333-fe0c11f312e0 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:08.674476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e048f51-3e70-4f23-8333-fe0c11f312e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:08.674995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e048f51-3e70-4f23-8333-fe0c11f312e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:08.675962 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e048f51-3e70-4f23-8333-fe0c11f312e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:08.676324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e048f51-3e70-4f23-8333-fe0c11f312e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:08.678266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e048f51-3e70-4f23-8333-fe0c11f312e0 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:08.696194 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 606/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:23:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:08.702582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd916964-47e9-4882-9029-1a522dd5305e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:08.788323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:08.834965 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd916964-47e9-4882-9029-1a522dd5305e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:08.889881 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237432b8-4e74-4a9a-bc31-689f957affdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:08.890603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237432b8-4e74-4a9a-bc31-689f957affdd None admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:08.891006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237432b8-4e74-4a9a-bc31-689f957affdd None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:08.891455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237432b8-4e74-4a9a-bc31-689f957affdd None admin] PATH_INFO: `/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/6db648aaa06b476e9a3cafbab832ee68/roles/248565f1fe7849c5b05a207eb214cbb7` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:08.916816 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237432b8-4e74-4a9a-bc31-689f957affdd None admin] RBAC: Authorizing `identity:create_grant(project_id=70a5f9cf0499405d82f5f5a47c2877eb, user_id=6db648aaa06b476e9a3cafbab832ee68, role_id=248565f1fe7849c5b05a207eb214cbb7)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:08.918388 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237432b8-4e74-4a9a-bc31-689f957affdd None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:08.933767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 626/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:23:08 2022] PUT /identity/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/6db648aaa06b476e9a3cafbab832ee68/roles/248565f1fe7849c5b05a207eb214cbb7 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:23:10.876243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018415df-fd47-4c80-8fef-1868b02f9fb5 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:10.876879 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018415df-fd47-4c80-8fef-1868b02f9fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:10.876879 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018415df-fd47-4c80-8fef-1868b02f9fb5 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:10.878220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 607/1233] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:10.884694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1e894-6caa-4344-8289-892e93d564a9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:10.885226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1e894-6caa-4344-8289-892e93d564a9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:10.885226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1e894-6caa-4344-8289-892e93d564a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:10.905183 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-38e1e894-6caa-4344-8289-892e93d564a9 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:10.906416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-38e1e894-6caa-4344-8289-892e93d564a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:10.992081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 627/1234] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:11.278495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018415df-fd47-4c80-8fef-1868b02f9fb5 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:11.308482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:11.406114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018415df-fd47-4c80-8fef-1868b02f9fb5 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:11.445471 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfeaff-31f2-47c5-bbb6-19f2070d7a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:11.446293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfeaff-31f2-47c5-bbb6-19f2070d7a39 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:11.446643 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfeaff-31f2-47c5-bbb6-19f2070d7a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:11.446959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfeaff-31f2-47c5-bbb6-19f2070d7a39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:11.458976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfeaff-31f2-47c5-bbb6-19f2070d7a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-e8dfeaff-31f2-47c5-bbb6-19f2070d7a39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:23:11.464248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:11.465331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:11.469565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 608/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 12:23:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:11.472635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1e894-6caa-4344-8289-892e93d564a9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:11.520996 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:11.581470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1e894-6caa-4344-8289-892e93d564a9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:11.622282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e262565-a276-4925-bf24-d049fc5272a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:11.622895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e262565-a276-4925-bf24-d049fc5272a0 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:11.623174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e262565-a276-4925-bf24-d049fc5272a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:11.623446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e262565-a276-4925-bf24-d049fc5272a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:11.624107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e262565-a276-4925-bf24-d049fc5272a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:11.624406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e262565-a276-4925-bf24-d049fc5272a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:11.625819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e262565-a276-4925-bf24-d049fc5272a0 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:11.634068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 628/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 12:23:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:23:11.640524 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfeaff-31f2-47c5-bbb6-19f2070d7a39 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:11.735685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:11.754695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfeaff-31f2-47c5-bbb6-19f2070d7a39 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:11.798773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f37d0-2818-4ac7-9f8a-74e761f655b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:11.799445 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f37d0-2818-4ac7-9f8a-74e761f655b7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:11.799693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f37d0-2818-4ac7-9f8a-74e761f655b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:11.800029 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f37d0-2818-4ac7-9f8a-74e761f655b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:11.812371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f37d0-2818-4ac7-9f8a-74e761f655b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:11.813971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f37d0-2818-4ac7-9f8a-74e761f655b7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-220f37d0-2818-4ac7-9f8a-74e761f655b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:11.832104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:11.836878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:11.838019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:11.838019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:11.838019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:11.838019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 609/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:11.839640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e262565-a276-4925-bf24-d049fc5272a0 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:11.954229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e262565-a276-4925-bf24-d049fc5272a0 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:11.964316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:11.994942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bcda0e-37b6-4d6b-838d-6cda10ddd765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:11.995523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bcda0e-37b6-4d6b-838d-6cda10ddd765 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:11.995827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bcda0e-37b6-4d6b-838d-6cda10ddd765 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:11.996070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bcda0e-37b6-4d6b-838d-6cda10ddd765 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:11.996784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bcda0e-37b6-4d6b-838d-6cda10ddd765 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:11.997087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bcda0e-37b6-4d6b-838d-6cda10ddd765 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:11.998567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bcda0e-37b6-4d6b-838d-6cda10ddd765 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:12.011785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 629/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:12.018361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f37d0-2818-4ac7-9f8a-74e761f655b7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:12.130552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f37d0-2818-4ac7-9f8a-74e761f655b7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:12.168169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68314dcb-d7b3-486b-8c7f-cdcac107f80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:12.168848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68314dcb-d7b3-486b-8c7f-cdcac107f80e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:12.169119 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68314dcb-d7b3-486b-8c7f-cdcac107f80e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:12.169387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68314dcb-d7b3-486b-8c7f-cdcac107f80e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:12.172404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:12.179291 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68314dcb-d7b3-486b-8c7f-cdcac107f80e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:12.181036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68314dcb-d7b3-486b-8c7f-cdcac107f80e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-68314dcb-d7b3-486b-8c7f-cdcac107f80e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:23:12.193126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:12.194435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:12.195851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:12.195851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:12.195851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:23:12.195851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:23:12.195851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:23:12.195851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:23:12.195851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:23:12.195851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:23:12.195851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:12.195851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:12.195851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 610/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:12.200883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bcda0e-37b6-4d6b-838d-6cda10ddd765 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:12.316853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bcda0e-37b6-4d6b-838d-6cda10ddd765 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:12.355689 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530741a-83a2-40d4-898b-d34163cc3d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:12.356376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530741a-83a2-40d4-898b-d34163cc3d51 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:12.356674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530741a-83a2-40d4-898b-d34163cc3d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:12.357176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530741a-83a2-40d4-898b-d34163cc3d51 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:12.357928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530741a-83a2-40d4-898b-d34163cc3d51 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:12.358253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530741a-83a2-40d4-898b-d34163cc3d51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:12.359773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530741a-83a2-40d4-898b-d34163cc3d51 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:12.385719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 630/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 24 12:23:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:12.393895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68314dcb-d7b3-486b-8c7f-cdcac107f80e None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:12.513839 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68314dcb-d7b3-486b-8c7f-cdcac107f80e None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:12.552086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41545b5-7279-4c81-842f-333b4ce78024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:12.552775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41545b5-7279-4c81-842f-333b4ce78024 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:12.553024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41545b5-7279-4c81-842f-333b4ce78024 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:12.553292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41545b5-7279-4c81-842f-333b4ce78024 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:12.625369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:12.635823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41545b5-7279-4c81-842f-333b4ce78024 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:12.637466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41545b5-7279-4c81-842f-333b4ce78024 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:12.751704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 611/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:23:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:23:12.757901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530741a-83a2-40d4-898b-d34163cc3d51 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:12.787743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:12.873644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530741a-83a2-40d4-898b-d34163cc3d51 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:12.916584 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab10004-91cb-441a-a17d-df6dd2150029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:12.917284 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab10004-91cb-441a-a17d-df6dd2150029 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:12.917650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab10004-91cb-441a-a17d-df6dd2150029 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:12.917970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab10004-91cb-441a-a17d-df6dd2150029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:12.929433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab10004-91cb-441a-a17d-df6dd2150029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:12.931194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab10004-91cb-441a-a17d-df6dd2150029 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-eab10004-91cb-441a-a17d-df6dd2150029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:12.949048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:12.950157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:12.951076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:12.951076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:12.951076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:12.951076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 631/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:12.958250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41545b5-7279-4c81-842f-333b4ce78024 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:13.067613 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:13.074414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41545b5-7279-4c81-842f-333b4ce78024 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:13.114335 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87789cb5-2105-4373-956c-5bbf579fd820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:13.114953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87789cb5-2105-4373-956c-5bbf579fd820 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:13.115231 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87789cb5-2105-4373-956c-5bbf579fd820 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:13.115609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87789cb5-2105-4373-956c-5bbf579fd820 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:13.116341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87789cb5-2105-4373-956c-5bbf579fd820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:13.116756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87789cb5-2105-4373-956c-5bbf579fd820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:13.118278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87789cb5-2105-4373-956c-5bbf579fd820 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:13.136059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 612/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:13.137381 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab10004-91cb-441a-a17d-df6dd2150029 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:13.226692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:13.251610 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab10004-91cb-441a-a17d-df6dd2150029 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:13.302643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc17f7b-1f85-462b-952f-bf8fded57420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:13.303703 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc17f7b-1f85-462b-952f-bf8fded57420 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:13.303976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc17f7b-1f85-462b-952f-bf8fded57420 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:13.304239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc17f7b-1f85-462b-952f-bf8fded57420 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:13.312627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc17f7b-1f85-462b-952f-bf8fded57420 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:13.315012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc17f7b-1f85-462b-952f-bf8fded57420 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-6bc17f7b-1f85-462b-952f-bf8fded57420 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:23:13.328070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:13.329685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:13.330764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:23:13.330764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:23:13.330764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:23:13.330764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:23:13.330764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:23:13.330764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:23:13.330764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:13.330764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:13.330764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 632/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:23:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:13.336177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87789cb5-2105-4373-956c-5bbf579fd820 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:13.461144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87789cb5-2105-4373-956c-5bbf579fd820 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:13.497353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180db241-fd97-4ab5-96f5-39b6c2750d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:13.497887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180db241-fd97-4ab5-96f5-39b6c2750d1b None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:13.498244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180db241-fd97-4ab5-96f5-39b6c2750d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:13.498566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180db241-fd97-4ab5-96f5-39b6c2750d1b None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:13.499241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180db241-fd97-4ab5-96f5-39b6c2750d1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:13.499550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180db241-fd97-4ab5-96f5-39b6c2750d1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:13.501028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180db241-fd97-4ab5-96f5-39b6c2750d1b None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:13.513755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 613/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:23:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:13.520123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc17f7b-1f85-462b-952f-bf8fded57420 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:13.623015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc17f7b-1f85-462b-952f-bf8fded57420 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:13.660256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18988c6d-ad50-44ba-ab40-6137ba360df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:13.660878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18988c6d-ad50-44ba-ab40-6137ba360df1 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:13.661154 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18988c6d-ad50-44ba-ab40-6137ba360df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:13.661424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18988c6d-ad50-44ba-ab40-6137ba360df1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:13.662148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18988c6d-ad50-44ba-ab40-6137ba360df1 None admin] RBAC: Adding query filter params (role.id=248565f1fe7849c5b05a207eb214cbb7, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=6db648aaa06b476e9a3cafbab832ee68) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:13.662448 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18988c6d-ad50-44ba-ab40-6137ba360df1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:13.663880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18988c6d-ad50-44ba-ab40-6137ba360df1 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:13.670240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 633/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:23:13 2022] GET /identity/v3/role_assignments?user.id=6db648aaa06b476e9a3cafbab832ee68&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=248565f1fe7849c5b05a207eb214cbb7 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:15.609648 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15548cf-1acb-4c8a-92a3-083f8b0d28d2 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:15.610240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15548cf-1acb-4c8a-92a3-083f8b0d28d2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:15.610240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15548cf-1acb-4c8a-92a3-083f8b0d28d2 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:15.611523 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 614/1247] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:15.617267 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26beccc-f35b-45a0-98b6-21d8838a94a3 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:15.617758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26beccc-f35b-45a0-98b6-21d8838a94a3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:15.617859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26beccc-f35b-45a0-98b6-21d8838a94a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:15.625657 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:15.638787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c26beccc-f35b-45a0-98b6-21d8838a94a3 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:15.640001 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c26beccc-f35b-45a0-98b6-21d8838a94a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:15.715671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 634/1248] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:23:15.985603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15548cf-1acb-4c8a-92a3-083f8b0d28d2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.005679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:16.108196 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15548cf-1acb-4c8a-92a3-083f8b0d28d2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:16.152875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e8a50-3f46-4613-8cd6-f10c7c2631e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:16.153887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e8a50-3f46-4613-8cd6-f10c7c2631e9 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.154313 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e8a50-3f46-4613-8cd6-f10c7c2631e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.154667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e8a50-3f46-4613-8cd6-f10c7c2631e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.170352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e8a50-3f46-4613-8cd6-f10c7c2631e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-8f5e8a50-3f46-4613-8cd6-f10c7c2631e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:23:16.175447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:16.176443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:16.178466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 615/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:23:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:23:16.185064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26beccc-f35b-45a0-98b6-21d8838a94a3 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.295314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26beccc-f35b-45a0-98b6-21d8838a94a3 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:16.328977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:16.335243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83135cd1-e631-4765-862d-a2dfded2335d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:16.335861 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83135cd1-e631-4765-862d-a2dfded2335d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.336155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83135cd1-e631-4765-862d-a2dfded2335d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.336425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83135cd1-e631-4765-862d-a2dfded2335d None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.337162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83135cd1-e631-4765-862d-a2dfded2335d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:16.337463 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83135cd1-e631-4765-862d-a2dfded2335d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:16.338899 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83135cd1-e631-4765-862d-a2dfded2335d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:16.352931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 635/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:23:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:23:16.354471 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e8a50-3f46-4613-8cd6-f10c7c2631e9 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.457093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:16.490844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e8a50-3f46-4613-8cd6-f10c7c2631e9 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:16.528446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab726486-dd8a-491e-9ee3-907f7eeca0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:16.529156 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab726486-dd8a-491e-9ee3-907f7eeca0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.529496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab726486-dd8a-491e-9ee3-907f7eeca0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.529794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab726486-dd8a-491e-9ee3-907f7eeca0f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.541596 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab726486-dd8a-491e-9ee3-907f7eeca0f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:16.543194 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab726486-dd8a-491e-9ee3-907f7eeca0f6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-ab726486-dd8a-491e-9ee3-907f7eeca0f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:16.560106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:16.561306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:16.562334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:16.562334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:16.562334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:16.562334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 616/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:16.567897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83135cd1-e631-4765-862d-a2dfded2335d None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.685114 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83135cd1-e631-4765-862d-a2dfded2335d None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:16.722269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce0289-8195-4a05-bd30-fdb99e6ff714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:16.722877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce0289-8195-4a05-bd30-fdb99e6ff714 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.723164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce0289-8195-4a05-bd30-fdb99e6ff714 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.723437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce0289-8195-4a05-bd30-fdb99e6ff714 None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.724097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce0289-8195-4a05-bd30-fdb99e6ff714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:16.724371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce0289-8195-4a05-bd30-fdb99e6ff714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:16.725795 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce0289-8195-4a05-bd30-fdb99e6ff714 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:16.740225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 636/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:16.746465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab726486-dd8a-491e-9ee3-907f7eeca0f6 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.857693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab726486-dd8a-491e-9ee3-907f7eeca0f6 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:16.887380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:16.896931 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71e796-5266-40fa-8383-c8a6311dece7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:16.897657 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71e796-5266-40fa-8383-c8a6311dece7 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.897937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71e796-5266-40fa-8383-c8a6311dece7 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.898207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71e796-5266-40fa-8383-c8a6311dece7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.907091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71e796-5266-40fa-8383-c8a6311dece7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:16.908562 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71e796-5266-40fa-8383-c8a6311dece7 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-8e71e796-5266-40fa-8383-c8a6311dece7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:23:16.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:16.922816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:16.923747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:16.923747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:16.923747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:23:16.923747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:23:16.923747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:23:16.923747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:23:16.923747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:23:16.923747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:23:16.923747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:16.923747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:16.923747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 617/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:16.929234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce0289-8195-4a05-bd30-fdb99e6ff714 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.995061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:17.044590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce0289-8195-4a05-bd30-fdb99e6ff714 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:17.083900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11afebb-0dc0-4910-9516-5471de303735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:17.084719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11afebb-0dc0-4910-9516-5471de303735 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:17.085081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11afebb-0dc0-4910-9516-5471de303735 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:17.085419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11afebb-0dc0-4910-9516-5471de303735 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:17.086173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11afebb-0dc0-4910-9516-5471de303735 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:17.086542 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11afebb-0dc0-4910-9516-5471de303735 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:17.088139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11afebb-0dc0-4910-9516-5471de303735 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:17.108165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 637/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 24 12:23:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:17.114126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71e796-5266-40fa-8383-c8a6311dece7 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:17.222602 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71e796-5266-40fa-8383-c8a6311dece7 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:17.259793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d08bf-9b91-4aae-bc08-37579e87474a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:17.261717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d08bf-9b91-4aae-bc08-37579e87474a None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:17.262068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d08bf-9b91-4aae-bc08-37579e87474a None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:17.262466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d08bf-9b91-4aae-bc08-37579e87474a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:17.329595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:17.344081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d08bf-9b91-4aae-bc08-37579e87474a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:17.345601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d08bf-9b91-4aae-bc08-37579e87474a None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:17.472177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 618/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:23:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:23:17.477175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11afebb-0dc0-4910-9516-5471de303735 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:17.582226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11afebb-0dc0-4910-9516-5471de303735 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:17.595792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:17.618026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06e11d-5892-4e0f-9b1b-84af53537a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:17.618666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06e11d-5892-4e0f-9b1b-84af53537a7a None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:17.618944 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06e11d-5892-4e0f-9b1b-84af53537a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:17.619210 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06e11d-5892-4e0f-9b1b-84af53537a7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:17.630481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06e11d-5892-4e0f-9b1b-84af53537a7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:17.631981 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06e11d-5892-4e0f-9b1b-84af53537a7a None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-4e06e11d-5892-4e0f-9b1b-84af53537a7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:17.649406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:17.650626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:17.651687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:17.651687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:17.651687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:17.651687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 638/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:17.656750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d08bf-9b91-4aae-bc08-37579e87474a None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:17.761949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:17.777321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d08bf-9b91-4aae-bc08-37579e87474a None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:17.814115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af707b-aa4d-41dd-802b-80644ace9564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:17.814715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af707b-aa4d-41dd-802b-80644ace9564 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:17.815014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af707b-aa4d-41dd-802b-80644ace9564 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:17.815284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af707b-aa4d-41dd-802b-80644ace9564 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:17.815926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af707b-aa4d-41dd-802b-80644ace9564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:17.816220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af707b-aa4d-41dd-802b-80644ace9564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:17.817637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af707b-aa4d-41dd-802b-80644ace9564 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:17.829829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 619/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:17.835944 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06e11d-5892-4e0f-9b1b-84af53537a7a None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:17.964266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06e11d-5892-4e0f-9b1b-84af53537a7a None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:18.006999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d970cb4-f887-4eee-b099-ef3887ee62bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:18.007630 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d970cb4-f887-4eee-b099-ef3887ee62bc None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:18.007945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d970cb4-f887-4eee-b099-ef3887ee62bc None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:18.008170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d970cb4-f887-4eee-b099-ef3887ee62bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:18.015347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d970cb4-f887-4eee-b099-ef3887ee62bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:18.016711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d970cb4-f887-4eee-b099-ef3887ee62bc None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-1d970cb4-f887-4eee-b099-ef3887ee62bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:23:18.028612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:18.029607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:18.030537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:23:18.030537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:23:18.030537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:23:18.030537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:23:18.030537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:23:18.030537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:23:18.030537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:18.030537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:18.030537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 639/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:23:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:23:18.036573 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af707b-aa4d-41dd-802b-80644ace9564 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:18.150370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af707b-aa4d-41dd-802b-80644ace9564 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:18.187538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175bc6f-568f-4baf-b64e-3c5f5038698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:18.188142 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175bc6f-568f-4baf-b64e-3c5f5038698e None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:18.188462 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175bc6f-568f-4baf-b64e-3c5f5038698e None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:18.188719 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175bc6f-568f-4baf-b64e-3c5f5038698e None admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:18.189396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175bc6f-568f-4baf-b64e-3c5f5038698e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:18.189689 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175bc6f-568f-4baf-b64e-3c5f5038698e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:18.191375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175bc6f-568f-4baf-b64e-3c5f5038698e None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:18.206795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 620/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:23:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:18.213536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d970cb4-f887-4eee-b099-ef3887ee62bc None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:18.243300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:18.325505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d970cb4-f887-4eee-b099-ef3887ee62bc None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:18.365027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00b397-782d-48cb-9f2e-e209390c7846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:18.365636 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00b397-782d-48cb-9f2e-e209390c7846 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:18.365916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00b397-782d-48cb-9f2e-e209390c7846 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:18.366221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00b397-782d-48cb-9f2e-e209390c7846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:18.366952 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00b397-782d-48cb-9f2e-e209390c7846 None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=6db648aaa06b476e9a3cafbab832ee68) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:18.367286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00b397-782d-48cb-9f2e-e209390c7846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:18.368803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00b397-782d-48cb-9f2e-e209390c7846 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:18.384783 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 640/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:23:18 2022] GET /identity/v3/role_assignments?user.id=6db648aaa06b476e9a3cafbab832ee68&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:20.334026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95d4f7-bacd-48a1-8281-408964f17345 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:20.334576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95d4f7-bacd-48a1-8281-408964f17345 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:20.334576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95d4f7-bacd-48a1-8281-408964f17345 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:20.336683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 621/1261] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:20.341398 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7fe9b-c7e5-4dc5-8670-9fe5346d33bb None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:20.341832 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7fe9b-c7e5-4dc5-8670-9fe5346d33bb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:20.341918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7fe9b-c7e5-4dc5-8670-9fe5346d33bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:20.362164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d9d7fe9b-c7e5-4dc5-8670-9fe5346d33bb None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:20.363464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d9d7fe9b-c7e5-4dc5-8670-9fe5346d33bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:20.440434 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 641/1262] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:20.704638 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95d4f7-bacd-48a1-8281-408964f17345 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:20.723665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:20.828384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95d4f7-bacd-48a1-8281-408964f17345 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:20.866947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745df6fe-9510-4183-b20f-9adf90bde616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:20.867562 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745df6fe-9510-4183-b20f-9adf90bde616 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:20.867837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745df6fe-9510-4183-b20f-9adf90bde616 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:20.868104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745df6fe-9510-4183-b20f-9adf90bde616 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:20.879531 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745df6fe-9510-4183-b20f-9adf90bde616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-745df6fe-9510-4183-b20f-9adf90bde616 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:23:20.884201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:20.885284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:20.886277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 622/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:23:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:20.892222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7fe9b-c7e5-4dc5-8670-9fe5346d33bb None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:20.914993 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:21.022014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7fe9b-c7e5-4dc5-8670-9fe5346d33bb None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:21.072240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6ea26-a81e-494f-94ce-254417dc2150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:21.073264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6ea26-a81e-494f-94ce-254417dc2150 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:21.073729 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6ea26-a81e-494f-94ce-254417dc2150 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:21.074167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6ea26-a81e-494f-94ce-254417dc2150 None admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:21.075236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6ea26-a81e-494f-94ce-254417dc2150 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:21.075731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6ea26-a81e-494f-94ce-254417dc2150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:21.078359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6ea26-a81e-494f-94ce-254417dc2150 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:21.091956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 642/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:23:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:23:21.100434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745df6fe-9510-4183-b20f-9adf90bde616 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:21.188370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:21.223683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745df6fe-9510-4183-b20f-9adf90bde616 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:21.258556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b2c93-bbca-45de-9c0b-ac5a202b051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:21.259111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b2c93-bbca-45de-9c0b-ac5a202b051f None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:21.259386 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b2c93-bbca-45de-9c0b-ac5a202b051f None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:21.259647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b2c93-bbca-45de-9c0b-ac5a202b051f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:21.271272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b2c93-bbca-45de-9c0b-ac5a202b051f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:21.272723 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b2c93-bbca-45de-9c0b-ac5a202b051f None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-282b2c93-bbca-45de-9c0b-ac5a202b051f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:21.288058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:21.289185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:21.290468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:21.290468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:21.290468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:21.290468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 623/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:21.295144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6ea26-a81e-494f-94ce-254417dc2150 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:21.390959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:21.418445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6ea26-a81e-494f-94ce-254417dc2150 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:21.456643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974e285-e3ca-4d3f-97ea-6eb6ac2ab0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:21.457274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974e285-e3ca-4d3f-97ea-6eb6ac2ab0da None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:21.457553 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974e285-e3ca-4d3f-97ea-6eb6ac2ab0da None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:21.457828 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974e285-e3ca-4d3f-97ea-6eb6ac2ab0da None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:21.458476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974e285-e3ca-4d3f-97ea-6eb6ac2ab0da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:21.458780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974e285-e3ca-4d3f-97ea-6eb6ac2ab0da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:21.460150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974e285-e3ca-4d3f-97ea-6eb6ac2ab0da None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:21.472111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 643/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:23:21.478114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b2c93-bbca-45de-9c0b-ac5a202b051f None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:21.591580 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b2c93-bbca-45de-9c0b-ac5a202b051f None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:21.612896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:21.630148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2519bd-de16-4a8a-aea6-8b0e68b156be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:21.630857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2519bd-de16-4a8a-aea6-8b0e68b156be None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:21.631137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2519bd-de16-4a8a-aea6-8b0e68b156be None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:21.631411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2519bd-de16-4a8a-aea6-8b0e68b156be None admin] PATH_INFO: `/v3/users/placement` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:21.639521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2519bd-de16-4a8a-aea6-8b0e68b156be None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:21.641091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2519bd-de16-4a8a-aea6-8b0e68b156be None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-fe2519bd-de16-4a8a-aea6-8b0e68b156be None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:23:21.653551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:21.654753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:21.655944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:21.655944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:21.655944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:23:21.655944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:23:21.655944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:23:21.655944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:23:21.655944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:23:21.655944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:23:21.655944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:21.655944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:21.655944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 624/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:21.662101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974e285-e3ca-4d3f-97ea-6eb6ac2ab0da None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:21.779801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974e285-e3ca-4d3f-97ea-6eb6ac2ab0da None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:21.820688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6363947-cf5a-41df-bae1-a18696f806b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:21.821305 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363947-cf5a-41df-bae1-a18696f806b4 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:21.821585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363947-cf5a-41df-bae1-a18696f806b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:21.821860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363947-cf5a-41df-bae1-a18696f806b4 None admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:21.822588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6363947-cf5a-41df-bae1-a18696f806b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:21.822892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6363947-cf5a-41df-bae1-a18696f806b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:21.824333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6363947-cf5a-41df-bae1-a18696f806b4 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:21.833601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:21.847499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 644/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 24 12:23:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:21.853425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2519bd-de16-4a8a-aea6-8b0e68b156be None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:21.966553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2519bd-de16-4a8a-aea6-8b0e68b156be None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:22.012723 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a31ca-ce32-4769-9e51-9237318ed4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:22.013373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a31ca-ce32-4769-9e51-9237318ed4fa None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:22.013643 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a31ca-ce32-4769-9e51-9237318ed4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:22.013909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a31ca-ce32-4769-9e51-9237318ed4fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:22.025788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a31ca-ce32-4769-9e51-9237318ed4fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:22.027241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a31ca-ce32-4769-9e51-9237318ed4fa None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-8d8a31ca-ce32-4769-9e51-9237318ed4fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:22.041780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:22.042831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:22.043736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:22.043736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:22.043736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:22.043736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 625/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:22.049273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6363947-cf5a-41df-bae1-a18696f806b4 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:22.160673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6363947-cf5a-41df-bae1-a18696f806b4 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:22.197840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c431df8-4d63-4011-b173-dafd0c8d0b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:22.198441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c431df8-4d63-4011-b173-dafd0c8d0b9e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:22.198717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c431df8-4d63-4011-b173-dafd0c8d0b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:22.198987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c431df8-4d63-4011-b173-dafd0c8d0b9e None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:22.199629 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c431df8-4d63-4011-b173-dafd0c8d0b9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:22.199934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c431df8-4d63-4011-b173-dafd0c8d0b9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:22.201348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c431df8-4d63-4011-b173-dafd0c8d0b9e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:22.213595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 645/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:22.222743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a31ca-ce32-4769-9e51-9237318ed4fa None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:22.263158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:22.334227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a31ca-ce32-4769-9e51-9237318ed4fa None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:22.373543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a399d-c93b-4da8-b7c6-3b0e82977ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:22.374172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a399d-c93b-4da8-b7c6-3b0e82977ccc None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:22.374485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a399d-c93b-4da8-b7c6-3b0e82977ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:22.374733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a399d-c93b-4da8-b7c6-3b0e82977ccc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:22.382356 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a399d-c93b-4da8-b7c6-3b0e82977ccc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:22.383785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a399d-c93b-4da8-b7c6-3b0e82977ccc None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-b97a399d-c93b-4da8-b7c6-3b0e82977ccc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:23:22.396799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:22.398190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:22.399706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:23:22.399706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:23:22.399706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:23:22.399706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:23:22.399706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:23:22.399706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:23:22.399706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:22.399706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:22.399706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 626/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:23:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:22.404408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c431df8-4d63-4011-b173-dafd0c8d0b9e None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:22.466458 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:22.511347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c431df8-4d63-4011-b173-dafd0c8d0b9e None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:22.563687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050e932-3f76-42d4-91ba-d1d8a9a75394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:22.564492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050e932-3f76-42d4-91ba-d1d8a9a75394 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:22.564856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050e932-3f76-42d4-91ba-d1d8a9a75394 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:22.565129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050e932-3f76-42d4-91ba-d1d8a9a75394 None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:22.567254 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050e932-3f76-42d4-91ba-d1d8a9a75394 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:22.567559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050e932-3f76-42d4-91ba-d1d8a9a75394 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:22.569059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050e932-3f76-42d4-91ba-d1d8a9a75394 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:22.584132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 646/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:23:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:22.590536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a399d-c93b-4da8-b7c6-3b0e82977ccc None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:22.685331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:22.695791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a399d-c93b-4da8-b7c6-3b0e82977ccc None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:22.735686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed340666-2a1d-4636-97b0-6baf301ebae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:22.736268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed340666-2a1d-4636-97b0-6baf301ebae6 None admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:22.736542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed340666-2a1d-4636-97b0-6baf301ebae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:22.736845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed340666-2a1d-4636-97b0-6baf301ebae6 None admin] PATH_INFO: `/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/6db648aaa06b476e9a3cafbab832ee68/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:22.738971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed340666-2a1d-4636-97b0-6baf301ebae6 None admin] RBAC: Authorizing `identity:create_grant(project_id=70a5f9cf0499405d82f5f5a47c2877eb, user_id=6db648aaa06b476e9a3cafbab832ee68, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:22.740460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed340666-2a1d-4636-97b0-6baf301ebae6 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:22.757908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 627/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 12:23:22 2022] PUT /identity/v3/projects/70a5f9cf0499405d82f5f5a47c2877eb/users/6db648aaa06b476e9a3cafbab832ee68/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:23:24.627682 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e1abb-c8ee-4d5e-b2c7-866745ca280c None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:24.628347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e1abb-c8ee-4d5e-b2c7-866745ca280c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:24.628347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e1abb-c8ee-4d5e-b2c7-866745ca280c None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:24.629759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 647/1274] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:24.636075 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8019338-c98a-46db-ac99-ec86b50a98a2 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:24.636769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8019338-c98a-46db-ac99-ec86b50a98a2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:24.636880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8019338-c98a-46db-ac99-ec86b50a98a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:24.665479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a8019338-c98a-46db-ac99-ec86b50a98a2 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:24.666739 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a8019338-c98a-46db-ac99-ec86b50a98a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:24.738702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 628/1275] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:25.015900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e1abb-c8ee-4d5e-b2c7-866745ca280c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:25.139660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e1abb-c8ee-4d5e-b2c7-866745ca280c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:25.157722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:25.176695 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c999-2e0d-4753-9292-824dc8bfab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:25.177342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c999-2e0d-4753-9292-824dc8bfab95 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:25.177617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c999-2e0d-4753-9292-824dc8bfab95 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:25.177888 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c999-2e0d-4753-9292-824dc8bfab95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:25.188013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36c999-2e0d-4753-9292-824dc8bfab95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-fc36c999-2e0d-4753-9292-824dc8bfab95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:23:25.193426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:25.196793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:25.197896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 648/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 12:23:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:25.202025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8019338-c98a-46db-ac99-ec86b50a98a2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:25.301531 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:25.322769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8019338-c98a-46db-ac99-ec86b50a98a2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:25.359538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a0c49-7357-47dd-8ceb-211c41026658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:25.360146 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a0c49-7357-47dd-8ceb-211c41026658 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:25.360422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a0c49-7357-47dd-8ceb-211c41026658 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:25.360733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a0c49-7357-47dd-8ceb-211c41026658 None admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:25.361401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a0c49-7357-47dd-8ceb-211c41026658 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:25.361703 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a0c49-7357-47dd-8ceb-211c41026658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:25.363170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a0c49-7357-47dd-8ceb-211c41026658 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:25.373019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 629/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 12:23:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:25.381456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c999-2e0d-4753-9292-824dc8bfab95 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:25.504761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c999-2e0d-4753-9292-824dc8bfab95 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:25.543063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44059a2-6678-458c-889a-fe8821623dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:25.543693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44059a2-6678-458c-889a-fe8821623dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:25.543972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44059a2-6678-458c-889a-fe8821623dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:25.544245 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44059a2-6678-458c-889a-fe8821623dc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:25.555634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44059a2-6678-458c-889a-fe8821623dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:25.557053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44059a2-6678-458c-889a-fe8821623dc5 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-f44059a2-6678-458c-889a-fe8821623dc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:25.572455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:25.573578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:25.574635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:25.574635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:25.574635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:25.574635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 649/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:25.580326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a0c49-7357-47dd-8ceb-211c41026658 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:25.687793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a0c49-7357-47dd-8ceb-211c41026658 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:25.724488 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19e170-fe1a-4983-8eaf-a52c27cfa308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:25.725101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19e170-fe1a-4983-8eaf-a52c27cfa308 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:25.725375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19e170-fe1a-4983-8eaf-a52c27cfa308 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:25.725646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19e170-fe1a-4983-8eaf-a52c27cfa308 None admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:25.726280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19e170-fe1a-4983-8eaf-a52c27cfa308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:25.726594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19e170-fe1a-4983-8eaf-a52c27cfa308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:25.727959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19e170-fe1a-4983-8eaf-a52c27cfa308 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:25.740152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 630/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:25.746308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44059a2-6678-458c-889a-fe8821623dc5 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:25.780276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:25.860424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44059a2-6678-458c-889a-fe8821623dc5 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:25.896988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160aa193-b66a-44d0-9b83-ae438f6d8adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:25.897611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160aa193-b66a-44d0-9b83-ae438f6d8adc None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:25.897891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160aa193-b66a-44d0-9b83-ae438f6d8adc None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:25.898157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160aa193-b66a-44d0-9b83-ae438f6d8adc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:25.905528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160aa193-b66a-44d0-9b83-ae438f6d8adc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:25.906995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160aa193-b66a-44d0-9b83-ae438f6d8adc None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-160aa193-b66a-44d0-9b83-ae438f6d8adc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:23:25.917995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:25.919033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:25.920069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:25.920069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:25.920069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:23:25.920069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:23:25.920069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:23:25.920069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:23:25.920069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:23:25.920069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:23:25.920069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:25.920069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:25.920069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 650/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:25.927244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19e170-fe1a-4983-8eaf-a52c27cfa308 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:25.931420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:26.037217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19e170-fe1a-4983-8eaf-a52c27cfa308 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:26.073645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305737a-3ad6-4200-8c09-b0cfca0fc7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:26.074239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305737a-3ad6-4200-8c09-b0cfca0fc7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:26.074514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305737a-3ad6-4200-8c09-b0cfca0fc7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:26.074782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305737a-3ad6-4200-8c09-b0cfca0fc7a3 None admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:26.075516 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305737a-3ad6-4200-8c09-b0cfca0fc7a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:26.075817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305737a-3ad6-4200-8c09-b0cfca0fc7a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:26.077297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305737a-3ad6-4200-8c09-b0cfca0fc7a3 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:26.095667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 631/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 24 12:23:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:26.102341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160aa193-b66a-44d0-9b83-ae438f6d8adc None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:26.207674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:26.224306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160aa193-b66a-44d0-9b83-ae438f6d8adc None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:26.262300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b784911-f7e9-4a62-a14e-fb7f6e4fd369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:26.262898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b784911-f7e9-4a62-a14e-fb7f6e4fd369 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:26.263174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b784911-f7e9-4a62-a14e-fb7f6e4fd369 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:26.263444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b784911-f7e9-4a62-a14e-fb7f6e4fd369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:26.333992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b784911-f7e9-4a62-a14e-fb7f6e4fd369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:26.335441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b784911-f7e9-4a62-a14e-fb7f6e4fd369 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:26.432039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:26.444273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 651/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 12:23:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:23:26.450431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305737a-3ad6-4200-8c09-b0cfca0fc7a3 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:26.514176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:26.558000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305737a-3ad6-4200-8c09-b0cfca0fc7a3 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:26.595801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deee7ee-1b5c-4427-980f-cae71f746e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:26.596424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deee7ee-1b5c-4427-980f-cae71f746e29 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:26.596705 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deee7ee-1b5c-4427-980f-cae71f746e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:26.597003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deee7ee-1b5c-4427-980f-cae71f746e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:26.607929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deee7ee-1b5c-4427-980f-cae71f746e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:26.609418 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deee7ee-1b5c-4427-980f-cae71f746e29 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-7deee7ee-1b5c-4427-980f-cae71f746e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:26.624618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:26.625717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:26.626629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:26.626629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:26.626629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:26.626629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 632/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:26.633949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b784911-f7e9-4a62-a14e-fb7f6e4fd369 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:26.743325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b784911-f7e9-4a62-a14e-fb7f6e4fd369 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:26.779342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a33af-94e1-4b8e-a2ec-bf230d15484c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:26.779943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a33af-94e1-4b8e-a2ec-bf230d15484c None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:26.780137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a33af-94e1-4b8e-a2ec-bf230d15484c None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:26.780404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a33af-94e1-4b8e-a2ec-bf230d15484c None admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:26.781080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a33af-94e1-4b8e-a2ec-bf230d15484c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:26.781378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a33af-94e1-4b8e-a2ec-bf230d15484c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:26.782766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a33af-94e1-4b8e-a2ec-bf230d15484c None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:26.794090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 652/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:23:26.802996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deee7ee-1b5c-4427-980f-cae71f746e29 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:26.916738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deee7ee-1b5c-4427-980f-cae71f746e29 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:26.930735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:26.954658 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb882391-e5f8-43a3-a328-a693f108feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:26.955283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb882391-e5f8-43a3-a328-a693f108feca None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:26.955561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb882391-e5f8-43a3-a328-a693f108feca None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:26.955828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb882391-e5f8-43a3-a328-a693f108feca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:26.962752 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb882391-e5f8-43a3-a328-a693f108feca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:26.964214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb882391-e5f8-43a3-a328-a693f108feca None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application [None req-cb882391-e5f8-43a3-a328-a693f108feca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:23:26.976598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:26.978293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:26.979845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:23:26.979845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:23:26.979845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:23:26.979845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:23:26.979845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:23:26.979845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:23:26.979845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:26.979845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: ERROR keystone.server.flask.application Aug 24 12:23:26.979845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 633/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 12:23:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:26.984445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a33af-94e1-4b8e-a2ec-bf230d15484c None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:27.039190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:27.099221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a33af-94e1-4b8e-a2ec-bf230d15484c None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:27.136472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351d712-032a-462c-a7dc-44006fd1e32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:27.137091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351d712-032a-462c-a7dc-44006fd1e32e None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:27.137369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351d712-032a-462c-a7dc-44006fd1e32e None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:27.137634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351d712-032a-462c-a7dc-44006fd1e32e None admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:27.138313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351d712-032a-462c-a7dc-44006fd1e32e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:27.138610 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351d712-032a-462c-a7dc-44006fd1e32e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:27.140039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351d712-032a-462c-a7dc-44006fd1e32e None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:27.151868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 653/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 12:23:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:27.158446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb882391-e5f8-43a3-a328-a693f108feca None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:27.262650 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb882391-e5f8-43a3-a328-a693f108feca None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:27.297605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27526d-24cf-4219-83f1-bbe1458ad3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:27.298113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27526d-24cf-4219-83f1-bbe1458ad3dd None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:27.298389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27526d-24cf-4219-83f1-bbe1458ad3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:27.298656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27526d-24cf-4219-83f1-bbe1458ad3dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:27.299374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27526d-24cf-4219-83f1-bbe1458ad3dd None admin] RBAC: Adding query filter params (role.id=c541c0c93cc7434ab3420744ede203a9, scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb, user.id=6db648aaa06b476e9a3cafbab832ee68) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:27.299666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27526d-24cf-4219-83f1-bbe1458ad3dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:27.301244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27526d-24cf-4219-83f1-bbe1458ad3dd None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:27.307203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 634/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 12:23:27 2022] GET /identity/v3/role_assignments?user.id=6db648aaa06b476e9a3cafbab832ee68&scope.project.id=70a5f9cf0499405d82f5f5a47c2877eb&role.id=c541c0c93cc7434ab3420744ede203a9 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:29.278922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de16e37-6c86-4690-9aa3-29d8a049831a None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:29.279460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de16e37-6c86-4690-9aa3-29d8a049831a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:29.279460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de16e37-6c86-4690-9aa3-29d8a049831a None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:29.280714 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 654/1288] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:29.287144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbf02d-5827-4f6c-9052-39fbcbb7c2fb None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:29.287579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbf02d-5827-4f6c-9052-39fbcbb7c2fb None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:29.287725 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbf02d-5827-4f6c-9052-39fbcbb7c2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:29.317160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5fcbf02d-5827-4f6c-9052-39fbcbb7c2fb None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:29.318349 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5fcbf02d-5827-4f6c-9052-39fbcbb7c2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:29.352628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:29.402081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 635/1289] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:29.667973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de16e37-6c86-4690-9aa3-29d8a049831a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:29.781181 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de16e37-6c86-4690-9aa3-29d8a049831a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:29.798112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:29.825501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d9764-d6f6-4609-bf5f-496a07bb2672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:29.826124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d9764-d6f6-4609-bf5f-496a07bb2672 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:29.826400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d9764-d6f6-4609-bf5f-496a07bb2672 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:29.826671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d9764-d6f6-4609-bf5f-496a07bb2672 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:29.827592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d9764-d6f6-4609-bf5f-496a07bb2672 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:29.828997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d9764-d6f6-4609-bf5f-496a07bb2672 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-b78d9764-d6f6-4609-bf5f-496a07bb2672 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:29.837906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:29.839065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:29.840292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 655/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 12:23:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:29.845245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbf02d-5827-4f6c-9052-39fbcbb7c2fb None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:29.959205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbf02d-5827-4f6c-9052-39fbcbb7c2fb None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:30.001929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cb986-7660-4685-adde-2b5d84e1abeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:30.002526 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cb986-7660-4685-adde-2b5d84e1abeb None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:30.002798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cb986-7660-4685-adde-2b5d84e1abeb None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:30.003066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cb986-7660-4685-adde-2b5d84e1abeb None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:30.003788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6cb986-7660-4685-adde-2b5d84e1abeb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:30.004083 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6cb986-7660-4685-adde-2b5d84e1abeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:30.005488 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6cb986-7660-4685-adde-2b5d84e1abeb None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:30.010513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 636/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Aug 24 12:23:29 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:30.016668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d9764-d6f6-4609-bf5f-496a07bb2672 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:30.126413 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d9764-d6f6-4609-bf5f-496a07bb2672 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:30.162962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28347f59-29fb-4891-8071-660b1b62cbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:30.163549 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28347f59-29fb-4891-8071-660b1b62cbaf None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:30.163824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28347f59-29fb-4891-8071-660b1b62cbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:30.164092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28347f59-29fb-4891-8071-660b1b62cbaf None admin] PATH_INFO: `/v3/services` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:30.164717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28347f59-29fb-4891-8071-660b1b62cbaf None admin] RBAC: Adding query filter params (type=placement) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:30.165049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28347f59-29fb-4891-8071-660b1b62cbaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:30.166421 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28347f59-29fb-4891-8071-660b1b62cbaf None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:30.171343 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 656/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Aug 24 12:23:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:32.064367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852394f-984b-46be-88d5-b5161a2d192f None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:32.064992 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852394f-984b-46be-88d5-b5161a2d192f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:32.064992 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852394f-984b-46be-88d5-b5161a2d192f None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:32.066909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 637/1293] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:32.073245 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cf0a3-698b-4397-9e48-ef04d685bc89 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:32.073696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cf0a3-698b-4397-9e48-ef04d685bc89 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:32.073894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cf0a3-698b-4397-9e48-ef04d685bc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:32.097823 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-283cf0a3-698b-4397-9e48-ef04d685bc89 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:32.099063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-283cf0a3-698b-4397-9e48-ef04d685bc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:32.132594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:32.175008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 657/1294] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:32.455428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852394f-984b-46be-88d5-b5161a2d192f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:32.489486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:32.579260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852394f-984b-46be-88d5-b5161a2d192f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:32.620901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3923f9-ea08-4650-912a-fd15f1d97643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:32.621504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3923f9-ea08-4650-912a-fd15f1d97643 None admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:32.621784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3923f9-ea08-4650-912a-fd15f1d97643 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:32.622059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3923f9-ea08-4650-912a-fd15f1d97643 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:32.622700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3923f9-ea08-4650-912a-fd15f1d97643 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:32.624128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3923f9-ea08-4650-912a-fd15f1d97643 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:32.646107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 638/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 12:23:32 2022] POST /identity/v3/services => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:23:34.547003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b08597-9224-4081-b148-58a8220f714c None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:34.547564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b08597-9224-4081-b148-58a8220f714c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:34.547564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b08597-9224-4081-b148-58a8220f714c None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:34.548926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 658/1296] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:34.558266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ae787-87ac-4833-a8f2-db3bea6af84e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:34.558993 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ae787-87ac-4833-a8f2-db3bea6af84e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:34.559148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ae787-87ac-4833-a8f2-db3bea6af84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:34.582557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-fa4ae787-87ac-4833-a8f2-db3bea6af84e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:34.583719 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fa4ae787-87ac-4833-a8f2-db3bea6af84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:34.662064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 639/1297] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:34.927307 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b08597-9224-4081-b148-58a8220f714c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:35.051273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b08597-9224-4081-b148-58a8220f714c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:35.086976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a258b5f-a813-45ab-aa41-60c95b01ccae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:35.087595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a258b5f-a813-45ab-aa41-60c95b01ccae None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:35.087873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a258b5f-a813-45ab-aa41-60c95b01ccae None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:35.088136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a258b5f-a813-45ab-aa41-60c95b01ccae None admin] PATH_INFO: `/v3/services/placement` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:35.088716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a258b5f-a813-45ab-aa41-60c95b01ccae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:35.090107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a258b5f-a813-45ab-aa41-60c95b01ccae None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-3a258b5f-a813-45ab-aa41-60c95b01ccae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:35.103366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:35.106731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:35.108038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 659/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 12:23:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:35.113863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ae787-87ac-4833-a8f2-db3bea6af84e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:35.134346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:35.237408 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ae787-87ac-4833-a8f2-db3bea6af84e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:35.278086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecdbad-ccbb-4714-be8e-f86ffa5ba894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:35.278682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecdbad-ccbb-4714-be8e-f86ffa5ba894 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:35.278954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecdbad-ccbb-4714-be8e-f86ffa5ba894 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:35.279220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecdbad-ccbb-4714-be8e-f86ffa5ba894 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:35.279841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecdbad-ccbb-4714-be8e-f86ffa5ba894 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:35.280142 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecdbad-ccbb-4714-be8e-f86ffa5ba894 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:35.281553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecdbad-ccbb-4714-be8e-f86ffa5ba894 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:35.287332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 640/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Aug 24 12:23:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:35.293613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a258b5f-a813-45ab-aa41-60c95b01ccae None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:35.329169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:35.412486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a258b5f-a813-45ab-aa41-60c95b01ccae None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:35.455710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c82354-757f-4371-b6a6-302e1f45456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:35.456449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c82354-757f-4371-b6a6-302e1f45456d None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:35.456778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c82354-757f-4371-b6a6-302e1f45456d None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:35.457099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c82354-757f-4371-b6a6-302e1f45456d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:35.457909 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c82354-757f-4371-b6a6-302e1f45456d None admin] RBAC: Adding query filter params (interface=public, service_id=bb71ff845b02428e806da9fa7c8198f0, region_id=RegionOne) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:35.458239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c82354-757f-4371-b6a6-302e1f45456d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:35.460008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c82354-757f-4371-b6a6-302e1f45456d None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:35.465610 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 660/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 12:23:35 2022] GET /identity/v3/endpoints?service_id=bb71ff845b02428e806da9fa7c8198f0&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:35.471634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecdbad-ccbb-4714-be8e-f86ffa5ba894 None admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:35.568389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:35.597271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecdbad-ccbb-4714-be8e-f86ffa5ba894 None admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:35.640703 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a25be1-3453-49f8-a849-6ead1debed11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:35.641318 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a25be1-3453-49f8-a849-6ead1debed11 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:35.641595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a25be1-3453-49f8-a849-6ead1debed11 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:35.641861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a25be1-3453-49f8-a849-6ead1debed11 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:35.642465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a25be1-3453-49f8-a849-6ead1debed11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:35.644063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a25be1-3453-49f8-a849-6ead1debed11 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:35.651136 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 641/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 12:23:35 2022] GET /identity/v3/services => generated 2191 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:23:37.529027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291a1c8-ff18-4af0-88d9-203bb4b1e05f None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:37.529931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291a1c8-ff18-4af0-88d9-203bb4b1e05f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:37.529931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291a1c8-ff18-4af0-88d9-203bb4b1e05f None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:37.532051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 661/1302] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:23:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:37.538776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f3940-ed97-4f11-b152-eb959ee5a194 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:37.539245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f3940-ed97-4f11-b152-eb959ee5a194 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:37.539339 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f3940-ed97-4f11-b152-eb959ee5a194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:37.558308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-973f3940-ed97-4f11-b152-eb959ee5a194 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:37.559495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-973f3940-ed97-4f11-b152-eb959ee5a194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:37.628489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 642/1303] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:37 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:23:37.891976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291a1c8-ff18-4af0-88d9-203bb4b1e05f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:38.002045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:38.009484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291a1c8-ff18-4af0-88d9-203bb4b1e05f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:38.043976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36ee1a-d366-401a-b070-c266180c64a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:38.044601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36ee1a-d366-401a-b070-c266180c64a7 None admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:38.044912 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36ee1a-d366-401a-b070-c266180c64a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:38.045182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36ee1a-d366-401a-b070-c266180c64a7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:38.045760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36ee1a-d366-401a-b070-c266180c64a7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:38.047138 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36ee1a-d366-401a-b070-c266180c64a7 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application [None req-fb36ee1a-d366-401a-b070-c266180c64a7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:38.056303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:38.057460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: ERROR keystone.server.flask.application Aug 24 12:23:38.058675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 662/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 12:23:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:38.066621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f3940-ed97-4f11-b152-eb959ee5a194 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:38.159842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:38.188110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f3940-ed97-4f11-b152-eb959ee5a194 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:38.228678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa736779-73fa-4116-b8ef-61c4dc2e89a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:38.229295 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa736779-73fa-4116-b8ef-61c4dc2e89a5 None admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:38.229568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa736779-73fa-4116-b8ef-61c4dc2e89a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:38.229835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa736779-73fa-4116-b8ef-61c4dc2e89a5 None admin] PATH_INFO: `/v3/services` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:38.230465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa736779-73fa-4116-b8ef-61c4dc2e89a5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:38.230769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa736779-73fa-4116-b8ef-61c4dc2e89a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:38.232153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa736779-73fa-4116-b8ef-61c4dc2e89a5 None admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:38.237421 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 643/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Aug 24 12:23:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:38.243604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36ee1a-d366-401a-b070-c266180c64a7 None admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:38.365737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36ee1a-d366-401a-b070-c266180c64a7 None admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:38.404760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23150631-74b3-4042-bdf1-3a657c04ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:38.405362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23150631-74b3-4042-bdf1-3a657c04ab87 None admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:38.405646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23150631-74b3-4042-bdf1-3a657c04ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:38.405915 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23150631-74b3-4042-bdf1-3a657c04ab87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:38.406580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23150631-74b3-4042-bdf1-3a657c04ab87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:38.407986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23150631-74b3-4042-bdf1-3a657c04ab87 None admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:38.430403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 663/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:23:38 2022] POST /identity/v3/endpoints => generated 331 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:23:48.234373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=30 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:48.425409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:43.223441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995013e-e353-4488-a897-953b74da1b31 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:43.224506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995013e-e353-4488-a897-953b74da1b31 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:43.225029 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995013e-e353-4488-a897-953b74da1b31 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:43.230797 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 644/1307] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:24:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:43.232120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c045f-f508-4af4-8ae3-1a1947142231 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:43.232560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c045f-f508-4af4-8ae3-1a1947142231 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:43.232678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c045f-f508-4af4-8ae3-1a1947142231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:43.318167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-893c045f-f508-4af4-8ae3-1a1947142231 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:43.319495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-893c045f-f508-4af4-8ae3-1a1947142231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:43.468694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 664/1308] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:24:43.789409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883c718-65ef-4e1d-9cb8-60ddbdb115ab None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:43.789713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883c718-65ef-4e1d-9cb8-60ddbdb115ab None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:43.789941 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883c718-65ef-4e1d-9cb8-60ddbdb115ab None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:43.791487 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 645/1309] 10.222.0.27 () {54 vars in 965 bytes} [Wed Aug 24 12:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:43.798359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8beb6-b7c8-4417-907b-55374b8218f1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:43.798765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8beb6-b7c8-4417-907b-55374b8218f1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:43.799251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8beb6-b7c8-4417-907b-55374b8218f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:43.883330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c4d8beb6-b7c8-4417-907b-55374b8218f1 None None] MFA Rules not processed for user `5e18461c8cc4497eae7a83d9ad1448f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:43.884794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c4d8beb6-b7c8-4417-907b-55374b8218f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:43.908151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:43.988248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 665/1310] 10.222.0.27 () {58 vars in 1062 bytes} [Wed Aug 24 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:43.997442 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883c718-65ef-4e1d-9cb8-60ddbdb115ab None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:43.999302 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6883c718-65ef-4e1d-9cb8-60ddbdb115ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:44.121062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883c718-65ef-4e1d-9cb8-60ddbdb115ab None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:44.206712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54085b40-d409-4061-995d-6bec12633c69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:44.207336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54085b40-d409-4061-995d-6bec12633c69 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:44.207613 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54085b40-d409-4061-995d-6bec12633c69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:44.207886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54085b40-d409-4061-995d-6bec12633c69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:44.209649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-54085b40-d409-4061-995d-6bec12633c69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:44.274402 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:44.337885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54085b40-d409-4061-995d-6bec12633c69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:44.339444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54085b40-d409-4061-995d-6bec12633c69 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:44.360912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 646/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:46.535194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8baf4ab-f195-4503-9ff8-a0b48d27ef5b None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:46.535752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8baf4ab-f195-4503-9ff8-a0b48d27ef5b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:46.535752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8baf4ab-f195-4503-9ff8-a0b48d27ef5b None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:46.538645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 666/1312] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:46.543333 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c0f9-859e-4c6a-b281-0b7485180a13 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:46.543865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c0f9-859e-4c6a-b281-0b7485180a13 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:46.543865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3c0f9-859e-4c6a-b281-0b7485180a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:46.623691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-11c3c0f9-859e-4c6a-b281-0b7485180a13 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:46.624941 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-11c3c0f9-859e-4c6a-b281-0b7485180a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:46.728299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 647/1313] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:47.026959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc0270-ae21-4064-8d2d-b27f723120e9 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:47.027222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc0270-ae21-4064-8d2d-b27f723120e9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:47.027531 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc0270-ae21-4064-8d2d-b27f723120e9 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:47.028930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 667/1314] 10.222.0.27 () {54 vars in 965 bytes} [Wed Aug 24 12:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:47.036953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35cac7-6516-49e7-bb92-a4c00282050e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:47.037554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35cac7-6516-49e7-bb92-a4c00282050e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:47.038108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35cac7-6516-49e7-bb92-a4c00282050e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:47.052396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:47.134613 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5d35cac7-6516-49e7-bb92-a4c00282050e None None] MFA Rules not processed for user `5e18461c8cc4497eae7a83d9ad1448f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:47.135832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5d35cac7-6516-49e7-bb92-a4c00282050e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:47.215240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 648/1315] 10.222.0.27 () {58 vars in 1062 bytes} [Wed Aug 24 12:24:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:47.224807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc0270-ae21-4064-8d2d-b27f723120e9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:47.226381 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dfbc0270-ae21-4064-8d2d-b27f723120e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:47.337140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc0270-ae21-4064-8d2d-b27f723120e9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:47.413812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:47.416314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95110b-e4c3-4ae6-b612-e4bd8d1d16fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:47.417360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95110b-e4c3-4ae6-b612-e4bd8d1d16fa service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:47.418152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95110b-e4c3-4ae6-b612-e4bd8d1d16fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:47.418531 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95110b-e4c3-4ae6-b612-e4bd8d1d16fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:47.420606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cc95110b-e4c3-4ae6-b612-e4bd8d1d16fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:47.543164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95110b-e4c3-4ae6-b612-e4bd8d1d16fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:47.544727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95110b-e4c3-4ae6-b612-e4bd8d1d16fa service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:47.566527 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 668/1316] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:50.002745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afd3b6-294b-42be-995a-1e97443b1874 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:50.003283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afd3b6-294b-42be-995a-1e97443b1874 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:50.003283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afd3b6-294b-42be-995a-1e97443b1874 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:50.004709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 649/1317] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:50.010660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105a88-8812-4c9a-a296-948a6965b849 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:50.011121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105a88-8812-4c9a-a296-948a6965b849 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:50.011216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3105a88-8812-4c9a-a296-948a6965b849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:50.087861 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d3105a88-8812-4c9a-a296-948a6965b849 None None] MFA Rules not processed for user `8563b95970a84781849db7018a225599`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:50.089103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d3105a88-8812-4c9a-a296-948a6965b849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:50.146775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:50.194548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 669/1318] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:50.480685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afd3b6-294b-42be-995a-1e97443b1874 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:50.483205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e4afd3b6-294b-42be-995a-1e97443b1874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:50.600604 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afd3b6-294b-42be-995a-1e97443b1874 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:50.603545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:50.679291 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb89ddb-7360-4450-9548-650fd1e5784a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:50.679899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb89ddb-7360-4450-9548-650fd1e5784a service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:50.680174 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb89ddb-7360-4450-9548-650fd1e5784a service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:50.680443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb89ddb-7360-4450-9548-650fd1e5784a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:50.682094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3bb89ddb-7360-4450-9548-650fd1e5784a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:50.802169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb89ddb-7360-4450-9548-650fd1e5784a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:50.803571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb89ddb-7360-4450-9548-650fd1e5784a service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:50.823045 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 650/1319] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:53.325594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524250e-c1e0-4d81-a203-4176957b56c8 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:53.326874 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524250e-c1e0-4d81-a203-4176957b56c8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:53.326874 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524250e-c1e0-4d81-a203-4176957b56c8 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:53.329586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 670/1320] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:24:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:53.338471 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2249a-bd37-48f0-8078-3801c9c1937d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:53.338896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2249a-bd37-48f0-8078-3801c9c1937d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:53.339014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2249a-bd37-48f0-8078-3801c9c1937d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:53.389666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:53.417426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-03a2249a-bd37-48f0-8078-3801c9c1937d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:53.418776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-03a2249a-bd37-48f0-8078-3801c9c1937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:53.521134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 651/1321] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:53.804569 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524250e-c1e0-4d81-a203-4176957b56c8 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:53.807149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a524250e-c1e0-4d81-a203-4176957b56c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:53.918597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524250e-c1e0-4d81-a203-4176957b56c8 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:54.004053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabc7b3-2eac-4ae1-bdd7-41c25445404b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:54.004688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabc7b3-2eac-4ae1-bdd7-41c25445404b service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:54.004958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabc7b3-2eac-4ae1-bdd7-41c25445404b service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:54.005224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabc7b3-2eac-4ae1-bdd7-41c25445404b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:54.006800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8cabc7b3-2eac-4ae1-bdd7-41c25445404b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:54.013845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:54.125882 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cabc7b3-2eac-4ae1-bdd7-41c25445404b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:54.127329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cabc7b3-2eac-4ae1-bdd7-41c25445404b service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:54.147434 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 671/1322] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:56.725445 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577ae5c-fe6a-4344-a41c-8d670df134bc None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:56.726012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577ae5c-fe6a-4344-a41c-8d670df134bc None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:56.726012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577ae5c-fe6a-4344-a41c-8d670df134bc None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:56.727326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 652/1323] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:56.732678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc72d8e-e366-4408-933e-7a41600ede4d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:56.733117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc72d8e-e366-4408-933e-7a41600ede4d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:56.733314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc72d8e-e366-4408-933e-7a41600ede4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:56.802677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2cc72d8e-e366-4408-933e-7a41600ede4d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:56.803897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2cc72d8e-e366-4408-933e-7a41600ede4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:56.865429 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:56.907091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 672/1324] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:57.191277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577ae5c-fe6a-4344-a41c-8d670df134bc None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:57.193632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8577ae5c-fe6a-4344-a41c-8d670df134bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:57.312204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577ae5c-fe6a-4344-a41c-8d670df134bc None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:57.325453 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:57.392009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe9e15-e102-4bd9-98ed-5d225fefb37c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:57.392543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe9e15-e102-4bd9-98ed-5d225fefb37c service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:57.392852 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe9e15-e102-4bd9-98ed-5d225fefb37c service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:57.393122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe9e15-e102-4bd9-98ed-5d225fefb37c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:57.394749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-11fe9e15-e102-4bd9-98ed-5d225fefb37c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:57.512532 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe9e15-e102-4bd9-98ed-5d225fefb37c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:57.514027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe9e15-e102-4bd9-98ed-5d225fefb37c service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:57.533374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 653/1325] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:59.479865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad232035-c98d-4c67-a0c9-095d9b20c85f None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:59.480401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad232035-c98d-4c67-a0c9-095d9b20c85f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:59.480401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad232035-c98d-4c67-a0c9-095d9b20c85f None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:59.481683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 673/1326] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:59.487051 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb906e-b2b5-4e37-a184-45ad2977b0ef None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:59.487500 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb906e-b2b5-4e37-a184-45ad2977b0ef None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:59.487715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb906e-b2b5-4e37-a184-45ad2977b0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:59.541427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:59.564525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-52bb906e-b2b5-4e37-a184-45ad2977b0ef None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:59.565789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-52bb906e-b2b5-4e37-a184-45ad2977b0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:59.668288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 654/1327] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:59.949643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad232035-c98d-4c67-a0c9-095d9b20c85f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:59.951255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ad232035-c98d-4c67-a0c9-095d9b20c85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:00.063100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad232035-c98d-4c67-a0c9-095d9b20c85f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:00.173900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791f7ba-944b-490c-91f9-f94e7695edcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:00.174677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f7ba-944b-490c-91f9-f94e7695edcf service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:00.175097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f7ba-944b-490c-91f9-f94e7695edcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:00.175621 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f7ba-944b-490c-91f9-f94e7695edcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:00.177615 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b791f7ba-944b-490c-91f9-f94e7695edcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:00.199230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:00.300078 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791f7ba-944b-490c-91f9-f94e7695edcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:00.301567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791f7ba-944b-490c-91f9-f94e7695edcf service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:00.321749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 674/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:02.278512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcf69f-a79f-4e62-9345-43987db8c19f None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:02.279067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcf69f-a79f-4e62-9345-43987db8c19f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:02.279067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcf69f-a79f-4e62-9345-43987db8c19f None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:02.281079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 655/1329] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:02.286872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8033862-0e81-4bc8-b1cf-1d5e1ec88dea None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:02.287362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8033862-0e81-4bc8-b1cf-1d5e1ec88dea None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:02.287484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8033862-0e81-4bc8-b1cf-1d5e1ec88dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:02.359518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a8033862-0e81-4bc8-b1cf-1d5e1ec88dea None None] MFA Rules not processed for user `8563b95970a84781849db7018a225599`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:02.360754 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a8033862-0e81-4bc8-b1cf-1d5e1ec88dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:02.428904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:02.463624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 675/1330] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:02.745545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcf69f-a79f-4e62-9345-43987db8c19f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:02.747180 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3abcf69f-a79f-4e62-9345-43987db8c19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:02.861014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcf69f-a79f-4e62-9345-43987db8c19f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:02.887463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:02.958485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86494b40-982f-4740-a854-f5d878ca82c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:02.959103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86494b40-982f-4740-a854-f5d878ca82c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:02.959378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86494b40-982f-4740-a854-f5d878ca82c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:02.959645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86494b40-982f-4740-a854-f5d878ca82c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:02.961287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-86494b40-982f-4740-a854-f5d878ca82c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:03.077121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86494b40-982f-4740-a854-f5d878ca82c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:03.078649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86494b40-982f-4740-a854-f5d878ca82c8 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:03.099858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 656/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:05.421137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4395f1f-b449-4c98-bca5-ff5e2547c8ea None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:05.421729 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4395f1f-b449-4c98-bca5-ff5e2547c8ea None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:05.421729 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4395f1f-b449-4c98-bca5-ff5e2547c8ea None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:05.423041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 676/1332] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:05.429480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466d46d-2de2-47a6-bb8e-5b9b4c7ee20a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:05.430179 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466d46d-2de2-47a6-bb8e-5b9b4c7ee20a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:05.430327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466d46d-2de2-47a6-bb8e-5b9b4c7ee20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:05.501406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:05.504204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a466d46d-2de2-47a6-bb8e-5b9b4c7ee20a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:05.505948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a466d46d-2de2-47a6-bb8e-5b9b4c7ee20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:05.617389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 657/1333] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:05.897824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4395f1f-b449-4c98-bca5-ff5e2547c8ea None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:05.899355 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f4395f1f-b449-4c98-bca5-ff5e2547c8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:06.014310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4395f1f-b449-4c98-bca5-ff5e2547c8ea None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:06.096594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74b99b-971c-4eb8-ae35-a38cab87f33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:06.097238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74b99b-971c-4eb8-ae35-a38cab87f33b service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:06.097514 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74b99b-971c-4eb8-ae35-a38cab87f33b service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:06.097782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74b99b-971c-4eb8-ae35-a38cab87f33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:06.099392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0c74b99b-971c-4eb8-ae35-a38cab87f33b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:06.127833 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:06.225245 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74b99b-971c-4eb8-ae35-a38cab87f33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:06.226605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74b99b-971c-4eb8-ae35-a38cab87f33b service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:06.247279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 677/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:08.241912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c32ed2-298a-4f74-b7f9-769cca4e40a2 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:08.242443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c32ed2-298a-4f74-b7f9-769cca4e40a2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:08.242443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c32ed2-298a-4f74-b7f9-769cca4e40a2 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:08.243762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 658/1335] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:08.250173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3f7da-7e9d-4495-baf2-947d753957ba None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:08.250911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3f7da-7e9d-4495-baf2-947d753957ba None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:08.251055 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3f7da-7e9d-4495-baf2-947d753957ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:08.337290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-17c3f7da-7e9d-4495-baf2-947d753957ba None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:08.338663 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-17c3f7da-7e9d-4495-baf2-947d753957ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:08.432711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:08.461186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 678/1336] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:08.769478 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c32ed2-298a-4f74-b7f9-769cca4e40a2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:08.771032 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-10c32ed2-298a-4f74-b7f9-769cca4e40a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:08.896577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c32ed2-298a-4f74-b7f9-769cca4e40a2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:08.929068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:08.983010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe75e1-8971-47aa-a7b3-92c6b8ced0d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:08.983633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe75e1-8971-47aa-a7b3-92c6b8ced0d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:08.983908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe75e1-8971-47aa-a7b3-92c6b8ced0d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:08.984173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe75e1-8971-47aa-a7b3-92c6b8ced0d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:08.985865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ecbe75e1-8971-47aa-a7b3-92c6b8ced0d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:09.108053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe75e1-8971-47aa-a7b3-92c6b8ced0d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:09.109567 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe75e1-8971-47aa-a7b3-92c6b8ced0d9 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:09.129351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 659/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:11.460136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7608c-fddc-418a-9a8e-125f241d8cf4 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:11.461069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7608c-fddc-418a-9a8e-125f241d8cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:11.461069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7608c-fddc-418a-9a8e-125f241d8cf4 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:11.466026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 679/1338] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:11.470908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26a11f-afae-4e8d-b783-934e67deb3a8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:11.471375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26a11f-afae-4e8d-b783-934e67deb3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:11.471464 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26a11f-afae-4e8d-b783-934e67deb3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:11.542520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-cc26a11f-afae-4e8d-b783-934e67deb3a8 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:11.543747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cc26a11f-afae-4e8d-b783-934e67deb3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:11.552399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:11.648869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 660/1339] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:11.929303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7608c-fddc-418a-9a8e-125f241d8cf4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:11.931806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6cb7608c-fddc-418a-9a8e-125f241d8cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:12.071248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7608c-fddc-418a-9a8e-125f241d8cf4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:12.150783 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d28f6-1a45-4c57-b755-9b9cdf8d01a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:12.151380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d28f6-1a45-4c57-b755-9b9cdf8d01a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:12.151661 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d28f6-1a45-4c57-b755-9b9cdf8d01a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:12.151927 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d28f6-1a45-4c57-b755-9b9cdf8d01a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:12.153555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b00d28f6-1a45-4c57-b755-9b9cdf8d01a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:12.190707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:12.271893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d28f6-1a45-4c57-b755-9b9cdf8d01a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:12.273369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d28f6-1a45-4c57-b755-9b9cdf8d01a3 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:12.293644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 680/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:16.327795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62686eeb-f9cd-4304-925e-57cbbb437ccf None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:16.328393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62686eeb-f9cd-4304-925e-57cbbb437ccf None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:16.328393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62686eeb-f9cd-4304-925e-57cbbb437ccf None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:16.329813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 661/1341] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:16.335392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f83d2-43a6-4492-85eb-48cd936c7a1f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:16.335826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f83d2-43a6-4492-85eb-48cd936c7a1f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:16.335937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f83d2-43a6-4492-85eb-48cd936c7a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:16.406656 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9e5f83d2-43a6-4492-85eb-48cd936c7a1f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:16.407891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9e5f83d2-43a6-4492-85eb-48cd936c7a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:16.499417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:16.511793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 681/1342] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:16.792431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62686eeb-f9cd-4304-925e-57cbbb437ccf None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:16.794003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-62686eeb-f9cd-4304-925e-57cbbb437ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:16.904617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62686eeb-f9cd-4304-925e-57cbbb437ccf None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:16.945135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:16.987303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07bed2-3c92-426e-98e4-f470afd16488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:16.987905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07bed2-3c92-426e-98e4-f470afd16488 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:16.988183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07bed2-3c92-426e-98e4-f470afd16488 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:16.988446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07bed2-3c92-426e-98e4-f470afd16488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:16.990107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1d07bed2-3c92-426e-98e4-f470afd16488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:17.107138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07bed2-3c92-426e-98e4-f470afd16488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:17.108944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07bed2-3c92-426e-98e4-f470afd16488 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:17.128138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 662/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:19.716346 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7c3e2-549a-4b3f-a3de-e02c4131a593 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:19.716986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7c3e2-549a-4b3f-a3de-e02c4131a593 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:19.717122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7c3e2-549a-4b3f-a3de-e02c4131a593 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:19.718518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 682/1344] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:19.723848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fda93f-38e6-4364-8502-5f21dda65de1 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:19.724307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fda93f-38e6-4364-8502-5f21dda65de1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:19.724397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fda93f-38e6-4364-8502-5f21dda65de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:19.794931 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d4fda93f-38e6-4364-8502-5f21dda65de1 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:19.796145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d4fda93f-38e6-4364-8502-5f21dda65de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:19.815635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:19.899106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 663/1345] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:20.181721 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7c3e2-549a-4b3f-a3de-e02c4131a593 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:20.183281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d6f7c3e2-549a-4b3f-a3de-e02c4131a593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:20.290444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7c3e2-549a-4b3f-a3de-e02c4131a593 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:20.371196 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074b6b2-6755-407b-8bf5-bf27c2bf524c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:20.371909 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b6b2-6755-407b-8bf5-bf27c2bf524c service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:20.372313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b6b2-6755-407b-8bf5-bf27c2bf524c service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:20.372778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b6b2-6755-407b-8bf5-bf27c2bf524c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:20.374488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6074b6b2-6755-407b-8bf5-bf27c2bf524c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:20.424114 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:20.494055 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074b6b2-6755-407b-8bf5-bf27c2bf524c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:20.495540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074b6b2-6755-407b-8bf5-bf27c2bf524c service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:20.514740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 683/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:24.474304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792a4e7-8e14-400c-9a11-87c539212506 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:24.474815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792a4e7-8e14-400c-9a11-87c539212506 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:24.474815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792a4e7-8e14-400c-9a11-87c539212506 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:24.478965 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 664/1347] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:24.482057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6589681-caba-42ce-8556-69d9ef8b7072 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:24.482528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6589681-caba-42ce-8556-69d9ef8b7072 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:24.482624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6589681-caba-42ce-8556-69d9ef8b7072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:24.557349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f6589681-caba-42ce-8556-69d9ef8b7072 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:24.558499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f6589681-caba-42ce-8556-69d9ef8b7072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:24.662995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:24.665656 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 684/1348] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:25:24.946073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792a4e7-8e14-400c-9a11-87c539212506 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:24.947526 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2792a4e7-8e14-400c-9a11-87c539212506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:25.071831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792a4e7-8e14-400c-9a11-87c539212506 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:25.126872 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:25.155991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77995b8c-1bd0-4a9a-9307-804043f139e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:25.156608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77995b8c-1bd0-4a9a-9307-804043f139e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:25.156920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77995b8c-1bd0-4a9a-9307-804043f139e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:25.157185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77995b8c-1bd0-4a9a-9307-804043f139e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:25.158822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-77995b8c-1bd0-4a9a-9307-804043f139e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:25.281752 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77995b8c-1bd0-4a9a-9307-804043f139e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:25.283189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77995b8c-1bd0-4a9a-9307-804043f139e6 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:25.303521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 665/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:30.345456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b1642-339f-4cca-8803-e64699a7cd43 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:30.346180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b1642-339f-4cca-8803-e64699a7cd43 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:30.346458 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b1642-339f-4cca-8803-e64699a7cd43 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:30.347849 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 685/1350] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:25:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:30.353783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34353968-821e-49eb-8add-2f53299c18b7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:30.354257 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34353968-821e-49eb-8add-2f53299c18b7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:30.354341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34353968-821e-49eb-8add-2f53299c18b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:30.476082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-34353968-821e-49eb-8add-2f53299c18b7 None None] MFA Rules not processed for user `6db648aaa06b476e9a3cafbab832ee68`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:30.477486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-34353968-821e-49eb-8add-2f53299c18b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:30.496865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:30.602367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 666/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:25:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:34.727482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec123c1-458e-41e7-883f-6effd1e4e1d0 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:34.728029 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec123c1-458e-41e7-883f-6effd1e4e1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:34.728313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec123c1-458e-41e7-883f-6effd1e4e1d0 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:34.729987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 686/1352] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:34.736301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3231c-4876-43ac-b4e4-bcabef79c0a8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:34.737111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3231c-4876-43ac-b4e4-bcabef79c0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:34.737573 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3231c-4876-43ac-b4e4-bcabef79c0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:34.821433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7da3231c-4876-43ac-b4e4-bcabef79c0a8 None None] MFA Rules not processed for user `6db648aaa06b476e9a3cafbab832ee68`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:34.822924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7da3231c-4876-43ac-b4e4-bcabef79c0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:34.899936 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:34.942883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 667/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:36.338671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38744671-faa0-4141-8f23-3ba4ad077547 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:36.339237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38744671-faa0-4141-8f23-3ba4ad077547 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:36.339237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38744671-faa0-4141-8f23-3ba4ad077547 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:36.340493 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 687/1354] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:36.345959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7027e5-bcb8-4f64-9522-c21a830a8c21 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:36.346380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7027e5-bcb8-4f64-9522-c21a830a8c21 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:36.346610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7027e5-bcb8-4f64-9522-c21a830a8c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:36.429249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8a7027e5-bcb8-4f64-9522-c21a830a8c21 None None] MFA Rules not processed for user `6db648aaa06b476e9a3cafbab832ee68`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:36.430452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8a7027e5-bcb8-4f64-9522-c21a830a8c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:36.544056 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 668/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:25:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:39.793259 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df633e-0400-40df-b205-1d10dcecd882 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:39.793839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df633e-0400-40df-b205-1d10dcecd882 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:39.793839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df633e-0400-40df-b205-1d10dcecd882 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:39.795193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 688/1356] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:39.800530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52d873-757e-487c-8a60-2e8f1ea4147c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:39.800996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52d873-757e-487c-8a60-2e8f1ea4147c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:39.801089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52d873-757e-487c-8a60-2e8f1ea4147c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:39.844204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:39.888000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8f52d873-757e-487c-8a60-2e8f1ea4147c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:39.889244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8f52d873-757e-487c-8a60-2e8f1ea4147c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:40.032110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 669/1357] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:40.336764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f45afc-b101-44b2-8c86-4fcb5bb47b65 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:40.337258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f45afc-b101-44b2-8c86-4fcb5bb47b65 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:40.337713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f45afc-b101-44b2-8c86-4fcb5bb47b65 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:40.339030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 689/1358] 10.222.0.27 () {54 vars in 962 bytes} [Wed Aug 24 12:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:40.347125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d4c34-9b4d-4efc-ad8a-758bedfd13d7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:40.347406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d4c34-9b4d-4efc-ad8a-758bedfd13d7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:40.347688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d4c34-9b4d-4efc-ad8a-758bedfd13d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:40.430938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d96d4c34-9b4d-4efc-ad8a-758bedfd13d7 None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:40.432144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d96d4c34-9b4d-4efc-ad8a-758bedfd13d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:40.482093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:40.554468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 670/1359] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Aug 24 12:25:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:40.563227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f45afc-b101-44b2-8c86-4fcb5bb47b65 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:40.564780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b3f45afc-b101-44b2-8c86-4fcb5bb47b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:40.722039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f45afc-b101-44b2-8c86-4fcb5bb47b65 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:40.798465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:40.841221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93619587-1df6-40e1-91cb-07583ad4a06c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:40.841730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93619587-1df6-40e1-91cb-07583ad4a06c service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:40.841996 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93619587-1df6-40e1-91cb-07583ad4a06c service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:40.842272 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93619587-1df6-40e1-91cb-07583ad4a06c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:40.843855 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-93619587-1df6-40e1-91cb-07583ad4a06c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:40.934378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93619587-1df6-40e1-91cb-07583ad4a06c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:40.936006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93619587-1df6-40e1-91cb-07583ad4a06c service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:40.954968 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 690/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:43.065693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac566a42-4a29-435c-9d48-de69c084c259 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:43.066230 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac566a42-4a29-435c-9d48-de69c084c259 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:43.066230 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac566a42-4a29-435c-9d48-de69c084c259 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:43.071902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 671/1361] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:43.074221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc29ce-ca21-4eed-a4f7-0a63f6dfb9e9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:43.074803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc29ce-ca21-4eed-a4f7-0a63f6dfb9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:43.075417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc29ce-ca21-4eed-a4f7-0a63f6dfb9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:43.158288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-aecc29ce-ca21-4eed-a4f7-0a63f6dfb9e9 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:43.159679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-aecc29ce-ca21-4eed-a4f7-0a63f6dfb9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:43.205655 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:43.273380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 691/1362] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:43.576955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac566a42-4a29-435c-9d48-de69c084c259 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:43.576955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ac566a42-4a29-435c-9d48-de69c084c259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:43.753560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac566a42-4a29-435c-9d48-de69c084c259 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:43.766677 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:43.878647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aceca0-7877-4b66-9a7f-743b39e52bb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:43.879263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aceca0-7877-4b66-9a7f-743b39e52bb6 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:43.879540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aceca0-7877-4b66-9a7f-743b39e52bb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:43.879818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aceca0-7877-4b66-9a7f-743b39e52bb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:43.881483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-77aceca0-7877-4b66-9a7f-743b39e52bb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:43.985895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aceca0-7877-4b66-9a7f-743b39e52bb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:43.987593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aceca0-7877-4b66-9a7f-743b39e52bb6 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:44.007067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 672/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:46.264802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa85f20-c1c1-403c-9581-814b455270dd None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:46.265743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa85f20-c1c1-403c-9581-814b455270dd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:46.266000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa85f20-c1c1-403c-9581-814b455270dd None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:46.268961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 692/1364] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:46.277041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd19c9a-520f-464d-9750-a4c78f50a259 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:46.277591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd19c9a-520f-464d-9750-a4c78f50a259 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:46.277591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd19c9a-520f-464d-9750-a4c78f50a259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:46.295831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:46.359678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8fd19c9a-520f-464d-9750-a4c78f50a259 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:46.360968 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8fd19c9a-520f-464d-9750-a4c78f50a259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:46.468731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 673/1365] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:46.754897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ec212-bc5e-437c-a9b8-7b64cc580b8b None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:46.755328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ec212-bc5e-437c-a9b8-7b64cc580b8b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:46.755639 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ec212-bc5e-437c-a9b8-7b64cc580b8b None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:46.757302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 693/1366] 10.222.0.27 () {54 vars in 962 bytes} [Wed Aug 24 12:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:46.765626 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80342e-b5e5-4d5a-b5ca-42b99486726a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:46.765935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80342e-b5e5-4d5a-b5ca-42b99486726a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:46.766183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80342e-b5e5-4d5a-b5ca-42b99486726a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:46.862871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3e80342e-b5e5-4d5a-b5ca-42b99486726a None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:46.864425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3e80342e-b5e5-4d5a-b5ca-42b99486726a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:46.903572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:46.996468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 674/1367] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Aug 24 12:25:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:47.004463 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ec212-bc5e-437c-a9b8-7b64cc580b8b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:47.006231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-158ec212-bc5e-437c-a9b8-7b64cc580b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:47.163153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ec212-bc5e-437c-a9b8-7b64cc580b8b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:47.184973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:47.287576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cedd3-16b1-43ba-b4e5-b24475315df5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:47.288246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cedd3-16b1-43ba-b4e5-b24475315df5 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:47.288540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cedd3-16b1-43ba-b4e5-b24475315df5 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:47.288840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cedd3-16b1-43ba-b4e5-b24475315df5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:47.290495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-636cedd3-16b1-43ba-b4e5-b24475315df5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:47.385153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cedd3-16b1-43ba-b4e5-b24475315df5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:47.387008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cedd3-16b1-43ba-b4e5-b24475315df5 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:47.406637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 694/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:48.024795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4be467-8f5e-43ad-a440-608410ed490a None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:48.025438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4be467-8f5e-43ad-a440-608410ed490a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:48.025438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4be467-8f5e-43ad-a440-608410ed490a None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:48.027060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 675/1369] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:48.032546 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21014edc-dfa3-436f-8070-5d1bd8528f0e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:48.032870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21014edc-dfa3-436f-8070-5d1bd8528f0e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:48.033166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21014edc-dfa3-436f-8070-5d1bd8528f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:48.068024 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:48.110281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-21014edc-dfa3-436f-8070-5d1bd8528f0e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:48.111507 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-21014edc-dfa3-436f-8070-5d1bd8528f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:48.221907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 695/1370] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:48.827959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f084c1-9912-436c-af91-a106303fdc41 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:48.827959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f084c1-9912-436c-af91-a106303fdc41 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:48.827959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f084c1-9912-436c-af91-a106303fdc41 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:48.827959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 676/1371] 10.222.0.27 () {54 vars in 964 bytes} [Wed Aug 24 12:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:48.832192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3743f-5161-435f-be9d-5d75d26489a7 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:48.832803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3743f-5161-435f-be9d-5d75d26489a7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:48.832803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3743f-5161-435f-be9d-5d75d26489a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:48.918192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0af3743f-5161-435f-be9d-5d75d26489a7 None None] MFA Rules not processed for user `a4b53c9f4ff842e08ec41f53147d0431`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:48.919433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0af3743f-5161-435f-be9d-5d75d26489a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:48.971103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:49.011969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 696/1372] 10.222.0.27 () {58 vars in 1061 bytes} [Wed Aug 24 12:25:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:49.013940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f084c1-9912-436c-af91-a106303fdc41 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:49.015484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-00f084c1-9912-436c-af91-a106303fdc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:49.126947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f084c1-9912-436c-af91-a106303fdc41 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:49.175168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:49.215374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728efaf-9d7b-49c8-8a28-4d8c45365813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:49.215892 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728efaf-9d7b-49c8-8a28-4d8c45365813 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:49.216218 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728efaf-9d7b-49c8-8a28-4d8c45365813 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:49.216491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728efaf-9d7b-49c8-8a28-4d8c45365813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:49.218073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1728efaf-9d7b-49c8-8a28-4d8c45365813 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:49.336846 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728efaf-9d7b-49c8-8a28-4d8c45365813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:49.338317 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728efaf-9d7b-49c8-8a28-4d8c45365813 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:49.357948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c42dd-6c9f-41fa-9e18-3f7d4e1af5fe None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:49.358242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c42dd-6c9f-41fa-9e18-3f7d4e1af5fe None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:49.358520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c42dd-6c9f-41fa-9e18-3f7d4e1af5fe None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:49.360215 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 697/1373] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:49.361152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 677/1374] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:49.365138 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4acd6-ab09-4d32-8020-5ce97d987d64 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:49.365453 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4acd6-ab09-4d32-8020-5ce97d987d64 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:49.365727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4acd6-ab09-4d32-8020-5ce97d987d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:49.437163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e0f4acd6-ab09-4d32-8020-5ce97d987d64 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:49.438488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e0f4acd6-ab09-4d32-8020-5ce97d987d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:49.549442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 698/1375] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:49.833198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728efaf-9d7b-49c8-8a28-4d8c45365813 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:49.834449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1728efaf-9d7b-49c8-8a28-4d8c45365813 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:49.905426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:49.940055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728efaf-9d7b-49c8-8a28-4d8c45365813 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:50.067751 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1da61e-e70c-4ac3-86ca-bf7699d10ace service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:50.068358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1da61e-e70c-4ac3-86ca-bf7699d10ace service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:50.068631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1da61e-e70c-4ac3-86ca-bf7699d10ace service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:50.068924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1da61e-e70c-4ac3-86ca-bf7699d10ace service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:50.070578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9a1da61e-e70c-4ac3-86ca-bf7699d10ace service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:50.154127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:50.165774 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1da61e-e70c-4ac3-86ca-bf7699d10ace service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:50.167483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1da61e-e70c-4ac3-86ca-bf7699d10ace service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:50.185456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 678/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:52.130649 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9c58b-cf83-46a5-a206-3d570ae145a7 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:52.131230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9c58b-cf83-46a5-a206-3d570ae145a7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:52.131230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9c58b-cf83-46a5-a206-3d570ae145a7 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:52.132544 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 699/1377] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:52.137981 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008654-fb18-44a1-8afe-339e25e3066c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:52.138411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008654-fb18-44a1-8afe-339e25e3066c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:52.138505 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8008654-fb18-44a1-8afe-339e25e3066c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:52.211560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e8008654-fb18-44a1-8afe-339e25e3066c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:52.212736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e8008654-fb18-44a1-8afe-339e25e3066c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:52.319350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 679/1378] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:52.606126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9c58b-cf83-46a5-a206-3d570ae145a7 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:52.607644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ece9c58b-cf83-46a5-a206-3d570ae145a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:52.660760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:52.791491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9c58b-cf83-46a5-a206-3d570ae145a7 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:53.060370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546b511-a556-4118-a220-a5d99bf5ea07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:53.061494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546b511-a556-4118-a220-a5d99bf5ea07 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:53.061787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546b511-a556-4118-a220-a5d99bf5ea07 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:53.062037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546b511-a556-4118-a220-a5d99bf5ea07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:53.063669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3546b511-a556-4118-a220-a5d99bf5ea07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:53.077132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:53.162137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546b511-a556-4118-a220-a5d99bf5ea07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:53.163856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546b511-a556-4118-a220-a5d99bf5ea07 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:53.183461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 700/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:55.139794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a57ac-aaa6-45d2-80d1-57d4b0f7a9bf None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:55.140328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a57ac-aaa6-45d2-80d1-57d4b0f7a9bf None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:55.140626 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a57ac-aaa6-45d2-80d1-57d4b0f7a9bf None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:55.142223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 680/1380] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:55.147617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb671bb6-3588-4ace-90d6-6bbbda579a38 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:55.148059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb671bb6-3588-4ace-90d6-6bbbda579a38 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:55.148159 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb671bb6-3588-4ace-90d6-6bbbda579a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:55.221683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-eb671bb6-3588-4ace-90d6-6bbbda579a38 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:55.222896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-eb671bb6-3588-4ace-90d6-6bbbda579a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:55.316608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:55.331147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 701/1381] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:55.618873 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a57ac-aaa6-45d2-80d1-57d4b0f7a9bf None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:55.620494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6c1a57ac-aaa6-45d2-80d1-57d4b0f7a9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:55.701910 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:55.793725 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a57ac-aaa6-45d2-80d1-57d4b0f7a9bf None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:55.930065 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f73301-8295-4ca1-9466-7b43b9b249d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:55.930639 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f73301-8295-4ca1-9466-7b43b9b249d7 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:55.930923 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f73301-8295-4ca1-9466-7b43b9b249d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:55.931187 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f73301-8295-4ca1-9466-7b43b9b249d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:55.932996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-79f73301-8295-4ca1-9466-7b43b9b249d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:55.956037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:56.040354 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f73301-8295-4ca1-9466-7b43b9b249d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:56.042204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f73301-8295-4ca1-9466-7b43b9b249d7 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:56.062164 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 681/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:58.141647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84da5c-b732-4d4a-8fb9-81d2e1355b92 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:58.142241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84da5c-b732-4d4a-8fb9-81d2e1355b92 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:58.142363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84da5c-b732-4d4a-8fb9-81d2e1355b92 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:58.144009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 702/1383] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:25:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:58.149875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9f174-c937-45f4-ad52-f0d4071ad10b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:58.150665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9f174-c937-45f4-ad52-f0d4071ad10b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:58.150665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9f174-c937-45f4-ad52-f0d4071ad10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:58.226301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-fac9f174-c937-45f4-ad52-f0d4071ad10b None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:58.227607 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fac9f174-c937-45f4-ad52-f0d4071ad10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:58.344575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:58.349560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 682/1384] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:58.708614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84da5c-b732-4d4a-8fb9-81d2e1355b92 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:58.710357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4d84da5c-b732-4d4a-8fb9-81d2e1355b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:58.862535 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84da5c-b732-4d4a-8fb9-81d2e1355b92 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:58.933186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:58.987406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ba28b-20e6-4da7-ba4b-01cb2ec9a5e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:58.988024 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ba28b-20e6-4da7-ba4b-01cb2ec9a5e8 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:58.988292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ba28b-20e6-4da7-ba4b-01cb2ec9a5e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:58.988552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ba28b-20e6-4da7-ba4b-01cb2ec9a5e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:58.990194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b37ba28b-20e6-4da7-ba4b-01cb2ec9a5e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:59.105063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ba28b-20e6-4da7-ba4b-01cb2ec9a5e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:59.107185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ba28b-20e6-4da7-ba4b-01cb2ec9a5e8 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:59.130483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 703/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:26:01.159204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db478ed1-ffb4-417a-b63b-2698bebfeeea None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:01.159825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db478ed1-ffb4-417a-b63b-2698bebfeeea None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:01.159825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db478ed1-ffb4-417a-b63b-2698bebfeeea None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:01.161330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 683/1386] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:01.167229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d8b78-8a36-44b4-b771-372092f1e356 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:01.167680 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d8b78-8a36-44b4-b771-372092f1e356 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:01.167825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d8b78-8a36-44b4-b771-372092f1e356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:01.242159 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d06d8b78-8a36-44b4-b771-372092f1e356 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:01.243440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d06d8b78-8a36-44b4-b771-372092f1e356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:01.271533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:01.350769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 704/1387] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:26:01.648029 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db478ed1-ffb4-417a-b63b-2698bebfeeea None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:01.649695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-db478ed1-ffb4-417a-b63b-2698bebfeeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:01.811017 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db478ed1-ffb4-417a-b63b-2698bebfeeea None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:01.887665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:01.950300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f1120-948a-42b7-a582-d9a402ac0d7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:01.950917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f1120-948a-42b7-a582-d9a402ac0d7f service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:01.951191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f1120-948a-42b7-a582-d9a402ac0d7f service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:01.951459 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f1120-948a-42b7-a582-d9a402ac0d7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:01.953125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4c5f1120-948a-42b7-a582-d9a402ac0d7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:02.055634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f1120-948a-42b7-a582-d9a402ac0d7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:02.058028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f1120-948a-42b7-a582-d9a402ac0d7f service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:02.080456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 684/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:04.057521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f3490-6e56-4958-9edd-485cc7f26acf None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:04.058129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f3490-6e56-4958-9edd-485cc7f26acf None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:04.058129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f3490-6e56-4958-9edd-485cc7f26acf None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:04.059373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 705/1389] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:04.065286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474645a-a8ee-468a-b566-d86bd1455c74 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:04.065755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474645a-a8ee-468a-b566-d86bd1455c74 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:04.065841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474645a-a8ee-468a-b566-d86bd1455c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:04.135139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c474645a-a8ee-468a-b566-d86bd1455c74 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:04.136436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c474645a-a8ee-468a-b566-d86bd1455c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:04.175959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:04.242062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 685/1390] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:04.280339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28834c9-dccd-4d89-98c7-61de711f685a None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:04.280707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28834c9-dccd-4d89-98c7-61de711f685a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:04.280929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28834c9-dccd-4d89-98c7-61de711f685a None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:04.282442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 706/1391] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:04.287621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6eefd-ad81-47c8-a904-a3e6fee337ef None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:04.287907 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6eefd-ad81-47c8-a904-a3e6fee337ef None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:04.288213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6eefd-ad81-47c8-a904-a3e6fee337ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:04.357821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-66c6eefd-ad81-47c8-a904-a3e6fee337ef None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:04.359055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-66c6eefd-ad81-47c8-a904-a3e6fee337ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:04.461203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 686/1392] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:04.530640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28834c9-dccd-4d89-98c7-61de711f685a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:04.532218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d28834c9-dccd-4d89-98c7-61de711f685a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:04.684024 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28834c9-dccd-4d89-98c7-61de711f685a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:04.694033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:04.773542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc65a0-0cfe-4349-ad33-0fed399a54f3 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:04.773829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc65a0-0cfe-4349-ad33-0fed399a54f3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:04.774170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc65a0-0cfe-4349-ad33-0fed399a54f3 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:04.775869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 687/1393] 10.222.0.27 () {54 vars in 967 bytes} [Wed Aug 24 12:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:04.781217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541ce6b-aadc-48a3-b540-3a5cee32f677 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:04.781499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541ce6b-aadc-48a3-b540-3a5cee32f677 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:04.781770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541ce6b-aadc-48a3-b540-3a5cee32f677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:04.802635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:04.805810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acbb79-2778-4a31-945a-8a67a107dcc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:04.807003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acbb79-2778-4a31-945a-8a67a107dcc1 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:04.807284 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acbb79-2778-4a31-945a-8a67a107dcc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:04.807568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acbb79-2778-4a31-945a-8a67a107dcc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:04.809211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-26acbb79-2778-4a31-945a-8a67a107dcc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:04.861816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8541ce6b-aadc-48a3-b540-3a5cee32f677 None None] MFA Rules not processed for user `f94bfed415d24a9cb596b1fa92d5df97`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:04.863187 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8541ce6b-aadc-48a3-b540-3a5cee32f677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:04.901884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acbb79-2778-4a31-945a-8a67a107dcc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:04.903507 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acbb79-2778-4a31-945a-8a67a107dcc1 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:04.922217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 707/1394] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:04.942339 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 688/1395] 10.222.0.27 () {58 vars in 1064 bytes} [Wed Aug 24 12:26:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:04.949861 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acbb79-2778-4a31-945a-8a67a107dcc1 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:04.951140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-26acbb79-2778-4a31-945a-8a67a107dcc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:04.969336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:05.042580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acbb79-2778-4a31-945a-8a67a107dcc1 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:05.124673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b24355-546e-488b-bc36-c780bd1c3025 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94bfed415d24a9cb596b1fa92d5df97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:05.125376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24355-546e-488b-bc36-c780bd1c3025 service glance] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:05.125546 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24355-546e-488b-bc36-c780bd1c3025 service glance] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:05.125826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24355-546e-488b-bc36-c780bd1c3025 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:05.127436 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e7b24355-546e-488b-bc36-c780bd1c3025 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:05.223327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:05.243442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b24355-546e-488b-bc36-c780bd1c3025 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:05.245025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b24355-546e-488b-bc36-c780bd1c3025 service glance] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:05.264697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 708/1396] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:06.032615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e6c68-39e3-4c3b-a6ab-124356dc7dec None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:06.033293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e6c68-39e3-4c3b-a6ab-124356dc7dec None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:06.033293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e6c68-39e3-4c3b-a6ab-124356dc7dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:06.102738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-849e6c68-39e3-4c3b-a6ab-124356dc7dec None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:06.103961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-849e6c68-39e3-4c3b-a6ab-124356dc7dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:06.163455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:06.196810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 689/1397] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:26:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:06.247760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4309bae-8f7c-4c48-92bb-cac1b4cb6139 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:06.248227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4309bae-8f7c-4c48-92bb-cac1b4cb6139 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:06.248394 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4309bae-8f7c-4c48-92bb-cac1b4cb6139 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:06.252753 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 709/1398] 10.222.0.27 () {54 vars in 943 bytes} [Wed Aug 24 12:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:06.261044 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b7c4d-b704-422d-b1f1-28f66887369a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:06.261360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b7c4d-b704-422d-b1f1-28f66887369a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:06.261610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b7c4d-b704-422d-b1f1-28f66887369a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:06.343411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-071b7c4d-b704-422d-b1f1-28f66887369a None None] MFA Rules not processed for user `bfec040b09c9415b9e89cd28c67aa5df`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:06.344617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-071b7c4d-b704-422d-b1f1-28f66887369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:06.459548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 690/1399] 10.222.0.27 () {58 vars in 1040 bytes} [Wed Aug 24 12:26:06 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:26:06.470015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4309bae-8f7c-4c48-92bb-cac1b4cb6139 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:06.471542 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d4309bae-8f7c-4c48-92bb-cac1b4cb6139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:06.618392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4309bae-8f7c-4c48-92bb-cac1b4cb6139 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:06.684379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:06.731265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf9bc1-5bdb-40bd-88b1-040dcc4bcb47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:06.731863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9bc1-5bdb-40bd-88b1-040dcc4bcb47 service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:06.732139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9bc1-5bdb-40bd-88b1-040dcc4bcb47 service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:06.732409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9bc1-5bdb-40bd-88b1-040dcc4bcb47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:06.734095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-89bf9bc1-5bdb-40bd-88b1-040dcc4bcb47 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:06.821101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf9bc1-5bdb-40bd-88b1-040dcc4bcb47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:06.822885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf9bc1-5bdb-40bd-88b1-040dcc4bcb47 service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:06.831173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 710/1400] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:26:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:26:06.864802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee4938-61d0-4661-af4b-31c8998410d3 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:06.865181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee4938-61d0-4661-af4b-31c8998410d3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:06.865392 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee4938-61d0-4661-af4b-31c8998410d3 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:06.866722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 691/1401] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:26:06.872541 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17de92-810a-4f4b-ba71-e1a0ec53ad1f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:06.872960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17de92-810a-4f4b-ba71-e1a0ec53ad1f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:06.873162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17de92-810a-4f4b-ba71-e1a0ec53ad1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:06.955690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-da17de92-810a-4f4b-ba71-e1a0ec53ad1f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:06.957072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-da17de92-810a-4f4b-ba71-e1a0ec53ad1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:07.023153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:07.086018 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 711/1402] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:07.387992 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee4938-61d0-4661-af4b-31c8998410d3 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:07.389656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-74ee4938-61d0-4661-af4b-31c8998410d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:07.432029 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:07.557833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee4938-61d0-4661-af4b-31c8998410d3 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:07.672717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:07.681358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8720e-c8e2-4574-9b07-6085c9a02403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:07.681978 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8720e-c8e2-4574-9b07-6085c9a02403 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:07.682253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8720e-c8e2-4574-9b07-6085c9a02403 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:07.682550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8720e-c8e2-4574-9b07-6085c9a02403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:07.684172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9fd8720e-c8e2-4574-9b07-6085c9a02403 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:07.788562 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8720e-c8e2-4574-9b07-6085c9a02403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:07.790524 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8720e-c8e2-4574-9b07-6085c9a02403 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:07.811873 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 692/1403] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:09.882941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea37ae9-a394-4f3d-80cc-1fa9406f2ef4 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:09.882941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea37ae9-a394-4f3d-80cc-1fa9406f2ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:09.882941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea37ae9-a394-4f3d-80cc-1fa9406f2ef4 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:09.886138 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 712/1404] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:09.890637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f900fa4-138c-4699-99e9-ddf0732efa37 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:09.891127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f900fa4-138c-4699-99e9-ddf0732efa37 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:09.891210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f900fa4-138c-4699-99e9-ddf0732efa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:09.998603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6f900fa4-138c-4699-99e9-ddf0732efa37 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:10.000917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6f900fa4-138c-4699-99e9-ddf0732efa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:10.098935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:10.140261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 693/1405] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:10.434198 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea37ae9-a394-4f3d-80cc-1fa9406f2ef4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:10.435760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5ea37ae9-a394-4f3d-80cc-1fa9406f2ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:10.622188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea37ae9-a394-4f3d-80cc-1fa9406f2ef4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:10.656530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:10.750218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b597ce3-4896-4d2d-9249-7aaee0166446 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:10.750740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b597ce3-4896-4d2d-9249-7aaee0166446 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:10.753247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b597ce3-4896-4d2d-9249-7aaee0166446 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:10.753537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b597ce3-4896-4d2d-9249-7aaee0166446 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:10.755826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6b597ce3-4896-4d2d-9249-7aaee0166446 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:10.851919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b597ce3-4896-4d2d-9249-7aaee0166446 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:10.853705 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b597ce3-4896-4d2d-9249-7aaee0166446 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:10.872350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 713/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:12.914034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a7a7d-e04e-41e2-b4ba-cf280f81ff6b None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:12.914573 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a7a7d-e04e-41e2-b4ba-cf280f81ff6b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:12.914726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a7a7d-e04e-41e2-b4ba-cf280f81ff6b None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:12.916863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 694/1407] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:12.924200 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930cbff-72d8-4ca9-9f83-df3ebcf734bf None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:12.924781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930cbff-72d8-4ca9-9f83-df3ebcf734bf None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:12.925034 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930cbff-72d8-4ca9-9f83-df3ebcf734bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:12.985556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:13.010447 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0930cbff-72d8-4ca9-9f83-df3ebcf734bf None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:13.011776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0930cbff-72d8-4ca9-9f83-df3ebcf734bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:13.125070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 714/1408] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:13.409899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a7a7d-e04e-41e2-b4ba-cf280f81ff6b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:13.411489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-394a7a7d-e04e-41e2-b4ba-cf280f81ff6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:13.578881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a7a7d-e04e-41e2-b4ba-cf280f81ff6b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:13.637631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:13.708909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32e7a4-6c03-449f-ac35-e469dcae92d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:13.709435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32e7a4-6c03-449f-ac35-e469dcae92d1 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:13.709714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32e7a4-6c03-449f-ac35-e469dcae92d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:13.710009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32e7a4-6c03-449f-ac35-e469dcae92d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:13.711797 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ce32e7a4-6c03-449f-ac35-e469dcae92d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:13.812385 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32e7a4-6c03-449f-ac35-e469dcae92d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:13.814429 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32e7a4-6c03-449f-ac35-e469dcae92d1 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:13.836528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 695/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:23.122774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=49 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:23.833891 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=75 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:28.906511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f4b24-51cb-40ca-a910-f4502831d963 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:28.907124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f4b24-51cb-40ca-a910-f4502831d963 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:28.907124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f4b24-51cb-40ca-a910-f4502831d963 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:28.908467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 715/1410] 10.222.0.27 () {54 vars in 918 bytes} [Wed Aug 24 12:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:28.914592 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe164dab-3bf3-462e-944b-437d29883a32 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:28.915009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe164dab-3bf3-462e-944b-437d29883a32 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:28.915191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe164dab-3bf3-462e-944b-437d29883a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:29.001203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-fe164dab-3bf3-462e-944b-437d29883a32 None None] MFA Rules not processed for user `6db648aaa06b476e9a3cafbab832ee68`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:29.002447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fe164dab-3bf3-462e-944b-437d29883a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:29.117789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 696/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Aug 24 12:26:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:29.136057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37dccf-afb8-4a62-97d0-e9389718678a None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:29.136478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37dccf-afb8-4a62-97d0-e9389718678a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:29.136978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37dccf-afb8-4a62-97d0-e9389718678a None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:29.139184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 716/1412] 10.222.0.27 () {54 vars in 918 bytes} [Wed Aug 24 12:26:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:29.143866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfde53d-aee7-43ca-a2ac-3563e126f0cb None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:29.144146 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfde53d-aee7-43ca-a2ac-3563e126f0cb None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:29.144414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfde53d-aee7-43ca-a2ac-3563e126f0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:29.235693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:29.238991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2bfde53d-aee7-43ca-a2ac-3563e126f0cb None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:29.240831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2bfde53d-aee7-43ca-a2ac-3563e126f0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:29.372600 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 697/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Aug 24 12:26:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:31.805452 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acea9b4-45af-49de-b051-10641ed28260 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:31.806270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acea9b4-45af-49de-b051-10641ed28260 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:31.806270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acea9b4-45af-49de-b051-10641ed28260 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:31.808790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:31.817649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5017e-a084-4356-b520-87d68bcadd8c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:31.818616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5017e-a084-4356-b520-87d68bcadd8c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:31.818616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5017e-a084-4356-b520-87d68bcadd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:31.902618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9bf5017e-a084-4356-b520-87d68bcadd8c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:31.903883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9bf5017e-a084-4356-b520-87d68bcadd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:31.963281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:32.009944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:32.300307 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acea9b4-45af-49de-b051-10641ed28260 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:32.302756 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8acea9b4-45af-49de-b051-10641ed28260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:32.459677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acea9b4-45af-49de-b051-10641ed28260 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:32.543222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:32.585579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b0a0c-113f-4717-939c-95145220bd1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:32.586217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b0a0c-113f-4717-939c-95145220bd1c service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:32.586490 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b0a0c-113f-4717-939c-95145220bd1c service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:32.586767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b0a0c-113f-4717-939c-95145220bd1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:32.588441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9f9b0a0c-113f-4717-939c-95145220bd1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:32.683336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b0a0c-113f-4717-939c-95145220bd1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:32.685007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b0a0c-113f-4717-939c-95145220bd1c service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:32.706268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 718/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:33.623353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21af92-40c0-4374-b7f6-bfb259924ec9 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:33.624264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21af92-40c0-4374-b7f6-bfb259924ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:33.624264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21af92-40c0-4374-b7f6-bfb259924ec9 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:33.626434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:33.634708 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a92058-ac96-4f61-aaed-5ddbbce6a019 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:33.635146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a92058-ac96-4f61-aaed-5ddbbce6a019 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:33.635266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a92058-ac96-4f61-aaed-5ddbbce6a019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:33.722250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a4a92058-ac96-4f61-aaed-5ddbbce6a019 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:33.723466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a4a92058-ac96-4f61-aaed-5ddbbce6a019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:33.750512 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:33.825853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:33.979093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21af92-40c0-4374-b7f6-bfb259924ec9 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:33.980678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6d21af92-40c0-4374-b7f6-bfb259924ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:34.140404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21af92-40c0-4374-b7f6-bfb259924ec9 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:34.185452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:34.269741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c077b-26e7-45b1-9880-ea20102bba2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:34.270256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c077b-26e7-45b1-9880-ea20102bba2a service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:34.270528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c077b-26e7-45b1-9880-ea20102bba2a service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:34.270794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c077b-26e7-45b1-9880-ea20102bba2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:34.272435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-164c077b-26e7-45b1-9880-ea20102bba2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:34.373654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c077b-26e7-45b1-9880-ea20102bba2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:34.375366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c077b-26e7-45b1-9880-ea20102bba2a service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:34.395450 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 700/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:35.280691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787bdf5-9339-4796-a00f-ca6e601b2eda None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:35.281228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787bdf5-9339-4796-a00f-ca6e601b2eda None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:35.281228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787bdf5-9339-4796-a00f-ca6e601b2eda None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:35.282558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:35.287749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83242eb3-a9ca-4b65-84b8-17fade91158a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:35.288206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83242eb3-a9ca-4b65-84b8-17fade91158a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:35.288311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83242eb3-a9ca-4b65-84b8-17fade91158a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:35.350742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:35.381707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-83242eb3-a9ca-4b65-84b8-17fade91158a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:35.382981 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-83242eb3-a9ca-4b65-84b8-17fade91158a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:35.496973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:35.652519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787bdf5-9339-4796-a00f-ca6e601b2eda None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:35.655279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9787bdf5-9339-4796-a00f-ca6e601b2eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:35.821802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787bdf5-9339-4796-a00f-ca6e601b2eda None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:35.835894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:35.947953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2ad21-9c53-4953-8b85-b6d367bf8b26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:35.948865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2ad21-9c53-4953-8b85-b6d367bf8b26 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:35.948865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2ad21-9c53-4953-8b85-b6d367bf8b26 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:35.949108 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2ad21-9c53-4953-8b85-b6d367bf8b26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:35.950613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8bd2ad21-9c53-4953-8b85-b6d367bf8b26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:36.046425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2ad21-9c53-4953-8b85-b6d367bf8b26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:36.048047 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2ad21-9c53-4953-8b85-b6d367bf8b26 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:36.068478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 721/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:36.957275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b896eb-e0fd-498a-8f88-2e7ea79d15a7 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:36.957781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b896eb-e0fd-498a-8f88-2e7ea79d15a7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:36.957781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b896eb-e0fd-498a-8f88-2e7ea79d15a7 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:36.959794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:36.968184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2554a-c93a-4e6e-85a4-e7320b9fa7d6 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:36.968614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2554a-c93a-4e6e-85a4-e7320b9fa7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:36.968780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2554a-c93a-4e6e-85a4-e7320b9fa7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:36.991739 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:37.050496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8ef2554a-c93a-4e6e-85a4-e7320b9fa7d6 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:37.051710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8ef2554a-c93a-4e6e-85a4-e7320b9fa7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:37.151160 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:37.306374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b896eb-e0fd-498a-8f88-2e7ea79d15a7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:37.308779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c0b896eb-e0fd-498a-8f88-2e7ea79d15a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:37.442499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:37.478986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b896eb-e0fd-498a-8f88-2e7ea79d15a7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:37.615610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14267788-9709-4097-a78b-77323af6cb21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:37.616212 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14267788-9709-4097-a78b-77323af6cb21 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:37.616483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14267788-9709-4097-a78b-77323af6cb21 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:37.616784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14267788-9709-4097-a78b-77323af6cb21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:37.618396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-14267788-9709-4097-a78b-77323af6cb21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:37.696621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:37.711528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14267788-9709-4097-a78b-77323af6cb21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:37.713188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14267788-9709-4097-a78b-77323af6cb21 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:37.733047 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 703/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:38.604714 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ae9f0-9599-4a4e-a3b8-ba5c2541e9d8 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:38.605297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ae9f0-9599-4a4e-a3b8-ba5c2541e9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:38.605297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ae9f0-9599-4a4e-a3b8-ba5c2541e9d8 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:38.606758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:38.611933 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba264a-fd53-4f3a-8d97-e0bb50e5249a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:38.612389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba264a-fd53-4f3a-8d97-e0bb50e5249a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:38.612495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba264a-fd53-4f3a-8d97-e0bb50e5249a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:38.692002 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e5ba264a-fd53-4f3a-8d97-e0bb50e5249a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:38.693247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e5ba264a-fd53-4f3a-8d97-e0bb50e5249a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:38.798383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:38.957307 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ae9f0-9599-4a4e-a3b8-ba5c2541e9d8 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:38.958842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c60ae9f0-9599-4a4e-a3b8-ba5c2541e9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:39.064120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:39.130879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ae9f0-9599-4a4e-a3b8-ba5c2541e9d8 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:39.259071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2373d-5c3d-4769-9707-f35db0755e23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:39.259705 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2373d-5c3d-4769-9707-f35db0755e23 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:39.259980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2373d-5c3d-4769-9707-f35db0755e23 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:39.260245 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2373d-5c3d-4769-9707-f35db0755e23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:39.261863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-43e2373d-5c3d-4769-9707-f35db0755e23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:39.314579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:39.362879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2373d-5c3d-4769-9707-f35db0755e23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:39.364753 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2373d-5c3d-4769-9707-f35db0755e23 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:39.386633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 724/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:40.197350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d4a94-defe-4963-94af-ab8a7349f244 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:40.197875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d4a94-defe-4963-94af-ab8a7349f244 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:40.197875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d4a94-defe-4963-94af-ab8a7349f244 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:40.199226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:40.205065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8da5e-1d38-4dd2-8bce-911f002dbdb0 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:40.205476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8da5e-1d38-4dd2-8bce-911f002dbdb0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:40.205611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8da5e-1d38-4dd2-8bce-911f002dbdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:40.285278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-83f8da5e-1d38-4dd2-8bce-911f002dbdb0 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:40.286425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-83f8da5e-1d38-4dd2-8bce-911f002dbdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:40.398263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:40.553334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d4a94-defe-4963-94af-ab8a7349f244 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:40.554790 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-986d4a94-defe-4963-94af-ab8a7349f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:40.604836 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:40.704682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d4a94-defe-4963-94af-ab8a7349f244 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:40.850501 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45045167-f54a-40ca-b72f-fcfdad57e13d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:40.851122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45045167-f54a-40ca-b72f-fcfdad57e13d service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:40.851399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45045167-f54a-40ca-b72f-fcfdad57e13d service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:40.851663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45045167-f54a-40ca-b72f-fcfdad57e13d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:40.853344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-45045167-f54a-40ca-b72f-fcfdad57e13d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:40.860955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:40.960494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45045167-f54a-40ca-b72f-fcfdad57e13d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:40.962188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45045167-f54a-40ca-b72f-fcfdad57e13d service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:40.982629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 706/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:41.795564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a96d0-caad-4e03-b390-0011293f3302 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:41.796085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a96d0-caad-4e03-b390-0011293f3302 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:41.796085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a96d0-caad-4e03-b390-0011293f3302 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:41.798116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:41.803475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf0a7a-12c4-448c-a793-e8ca0c66a34d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:41.803964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf0a7a-12c4-448c-a793-e8ca0c66a34d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:41.804284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf0a7a-12c4-448c-a793-e8ca0c66a34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:41.887915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-00cf0a7a-12c4-448c-a793-e8ca0c66a34d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:41.889393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-00cf0a7a-12c4-448c-a793-e8ca0c66a34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:41.964742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:42.001097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:26:42.156313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a96d0-caad-4e03-b390-0011293f3302 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:42.158746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dd9a96d0-caad-4e03-b390-0011293f3302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:42.181321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:42.317454 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a96d0-caad-4e03-b390-0011293f3302 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:42.418490 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:42.446117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafc154-91a9-46c1-8bd4-fa14fb2e1040 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:42.446775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafc154-91a9-46c1-8bd4-fa14fb2e1040 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:42.447053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafc154-91a9-46c1-8bd4-fa14fb2e1040 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:42.447317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafc154-91a9-46c1-8bd4-fa14fb2e1040 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:42.448963 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bcafc154-91a9-46c1-8bd4-fa14fb2e1040 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:42.548504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafc154-91a9-46c1-8bd4-fa14fb2e1040 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:42.550163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafc154-91a9-46c1-8bd4-fa14fb2e1040 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:42.572229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 727/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:43.389840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f2c6b-6a43-4cff-afb7-8b5a0b576011 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:43.390476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f2c6b-6a43-4cff-afb7-8b5a0b576011 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:43.390476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f2c6b-6a43-4cff-afb7-8b5a0b576011 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:43.391747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:43.397088 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cc29a-7d90-4922-bf7f-864a188c1ae1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:43.397576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cc29a-7d90-4922-bf7f-864a188c1ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:43.397683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cc29a-7d90-4922-bf7f-864a188c1ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:43.489647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-911cc29a-7d90-4922-bf7f-864a188c1ae1 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:43.490906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-911cc29a-7d90-4922-bf7f-864a188c1ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:43.536757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:43.598696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:43.757899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f2c6b-6a43-4cff-afb7-8b5a0b576011 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:43.759432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-506f2c6b-6a43-4cff-afb7-8b5a0b576011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:43.918210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f2c6b-6a43-4cff-afb7-8b5a0b576011 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:43.986962 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:44.050082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe0bc0-a253-4414-a194-0ed91c6c1561 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:44.050682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe0bc0-a253-4414-a194-0ed91c6c1561 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:44.050954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe0bc0-a253-4414-a194-0ed91c6c1561 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:44.051219 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe0bc0-a253-4414-a194-0ed91c6c1561 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:44.052837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-efbe0bc0-a253-4414-a194-0ed91c6c1561 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:44.145492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe0bc0-a253-4414-a194-0ed91c6c1561 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:44.147144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe0bc0-a253-4414-a194-0ed91c6c1561 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:44.166599 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 709/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:44.989184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0c54d-825c-43f4-a2d6-2a6c34c5edf3 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:44.989945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0c54d-825c-43f4-a2d6-2a6c34c5edf3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:44.989945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0c54d-825c-43f4-a2d6-2a6c34c5edf3 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:44.992220 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:44.999837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57d84d-1438-4191-b874-5ae96c476190 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:45.000272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57d84d-1438-4191-b874-5ae96c476190 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:45.000385 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57d84d-1438-4191-b874-5ae96c476190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:45.083430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2e57d84d-1438-4191-b874-5ae96c476190 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:45.084625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2e57d84d-1438-4191-b874-5ae96c476190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:45.086823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:45.187489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:26:45.340094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0c54d-825c-43f4-a2d6-2a6c34c5edf3 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:45.342586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-85a0c54d-825c-43f4-a2d6-2a6c34c5edf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:45.519047 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0c54d-825c-43f4-a2d6-2a6c34c5edf3 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:45.547970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:45.646127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8808913-b7c2-4178-8355-a01a51cb5a3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:45.646731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8808913-b7c2-4178-8355-a01a51cb5a3d service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:45.647007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8808913-b7c2-4178-8355-a01a51cb5a3d service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:45.647280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8808913-b7c2-4178-8355-a01a51cb5a3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:45.648891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f8808913-b7c2-4178-8355-a01a51cb5a3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:45.758543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8808913-b7c2-4178-8355-a01a51cb5a3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:45.760151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8808913-b7c2-4178-8355-a01a51cb5a3d service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:45.780681 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 730/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:47.621110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150e19d-ff1a-48c5-9478-61250a1606da None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:47.621714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150e19d-ff1a-48c5-9478-61250a1606da None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:47.621714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150e19d-ff1a-48c5-9478-61250a1606da None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:47.623766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:47.633185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c22ea-a898-4ba4-88b0-161fb45df983 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:47.633986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c22ea-a898-4ba4-88b0-161fb45df983 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:47.634138 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c22ea-a898-4ba4-88b0-161fb45df983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:47.679572 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:47.713981 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4d8c22ea-a898-4ba4-88b0-161fb45df983 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:47.715211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4d8c22ea-a898-4ba4-88b0-161fb45df983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:47.823521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:48.119840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150e19d-ff1a-48c5-9478-61250a1606da None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:48.122396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9150e19d-ff1a-48c5-9478-61250a1606da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:48.250925 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150e19d-ff1a-48c5-9478-61250a1606da None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:48.286103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:48.331598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85584a-c52f-49eb-915c-d8381af7f528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:48.332210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85584a-c52f-49eb-915c-d8381af7f528 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:48.332487 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85584a-c52f-49eb-915c-d8381af7f528 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:48.332786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85584a-c52f-49eb-915c-d8381af7f528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:48.334432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cb85584a-c52f-49eb-915c-d8381af7f528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:48.465047 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85584a-c52f-49eb-915c-d8381af7f528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:48.466573 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85584a-c52f-49eb-915c-d8381af7f528 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:48.486934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 712/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:50.574178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd5f4a-6949-428e-8304-7abd193f15c3 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:50.574829 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd5f4a-6949-428e-8304-7abd193f15c3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:50.574829 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd5f4a-6949-428e-8304-7abd193f15c3 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:50.576172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:50.586028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b072b-f0ae-43a5-b43c-a9e3e260103e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:50.586793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b072b-f0ae-43a5-b43c-a9e3e260103e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:50.586949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b072b-f0ae-43a5-b43c-a9e3e260103e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:50.673384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-823b072b-f0ae-43a5-b43c-a9e3e260103e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:50.674611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-823b072b-f0ae-43a5-b43c-a9e3e260103e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:50.687681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:50.777115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:51.063568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd5f4a-6949-428e-8304-7abd193f15c3 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:51.065989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8cfd5f4a-6949-428e-8304-7abd193f15c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:51.186918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd5f4a-6949-428e-8304-7abd193f15c3 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:51.202897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:51.267611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74003a-9a70-43a6-8a16-6d63f554a695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:51.268220 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74003a-9a70-43a6-8a16-6d63f554a695 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:51.268499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74003a-9a70-43a6-8a16-6d63f554a695 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:51.268797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74003a-9a70-43a6-8a16-6d63f554a695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:51.270402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0a74003a-9a70-43a6-8a16-6d63f554a695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:51.387561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74003a-9a70-43a6-8a16-6d63f554a695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:51.389374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74003a-9a70-43a6-8a16-6d63f554a695 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:51.407995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 733/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:53.715938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd15687-0c57-491f-86e1-628c2a6254d6 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:53.716561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd15687-0c57-491f-86e1-628c2a6254d6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:53.716561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd15687-0c57-491f-86e1-628c2a6254d6 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:53.719457 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:53.728840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795d7cc-37d7-467e-875b-5b6748f7a614 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:53.729295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795d7cc-37d7-467e-875b-5b6748f7a614 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:53.729435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795d7cc-37d7-467e-875b-5b6748f7a614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:53.788005 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:53.807870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0795d7cc-37d7-467e-875b-5b6748f7a614 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:53.809129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0795d7cc-37d7-467e-875b-5b6748f7a614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:53.922178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:54.211478 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd15687-0c57-491f-86e1-628c2a6254d6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:54.213077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2fd15687-0c57-491f-86e1-628c2a6254d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:54.325486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd15687-0c57-491f-86e1-628c2a6254d6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:54.367883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:54.404621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d056eec-e4f0-4fc5-b51d-f4e4bb8cec69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:54.405332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d056eec-e4f0-4fc5-b51d-f4e4bb8cec69 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:54.405603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d056eec-e4f0-4fc5-b51d-f4e4bb8cec69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:54.405870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d056eec-e4f0-4fc5-b51d-f4e4bb8cec69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:54.407498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8d056eec-e4f0-4fc5-b51d-f4e4bb8cec69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:54.541050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d056eec-e4f0-4fc5-b51d-f4e4bb8cec69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:54.542413 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d056eec-e4f0-4fc5-b51d-f4e4bb8cec69 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:54.563559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 715/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:56.643387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d13050-a94d-400d-90d7-41dffac70c02 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:56.643995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d13050-a94d-400d-90d7-41dffac70c02 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:56.643995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d13050-a94d-400d-90d7-41dffac70c02 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:56.646077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:56.655884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9c8ac-cf16-47d8-be8a-44d1dea52708 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:56.656325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9c8ac-cf16-47d8-be8a-44d1dea52708 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:56.656444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9c8ac-cf16-47d8-be8a-44d1dea52708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:56.732041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6ec9c8ac-cf16-47d8-be8a-44d1dea52708 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:56.733393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6ec9c8ac-cf16-47d8-be8a-44d1dea52708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:56.764473 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:56.852007 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:57.144037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d13050-a94d-400d-90d7-41dffac70c02 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:57.145635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a7d13050-a94d-400d-90d7-41dffac70c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:57.259604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d13050-a94d-400d-90d7-41dffac70c02 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:57.280802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:57.354288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1fabf-f30a-4a79-90a3-25a4ce5421e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:57.355161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1fabf-f30a-4a79-90a3-25a4ce5421e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:57.355608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1fabf-f30a-4a79-90a3-25a4ce5421e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:57.356031 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1fabf-f30a-4a79-90a3-25a4ce5421e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:57.358824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-eec1fabf-f30a-4a79-90a3-25a4ce5421e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:57.481490 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1fabf-f30a-4a79-90a3-25a4ce5421e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:57.482931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1fabf-f30a-4a79-90a3-25a4ce5421e5 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:57.501817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 736/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:26:59.603523 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21d9f3-1c54-4255-aa9b-1aca06349163 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:59.604404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21d9f3-1c54-4255-aa9b-1aca06349163 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:59.604404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21d9f3-1c54-4255-aa9b-1aca06349163 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:59.606572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:59.614367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b24a4-eec6-4d9a-81d9-57c110d05457 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:59.615129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b24a4-eec6-4d9a-81d9-57c110d05457 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:59.615273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b24a4-eec6-4d9a-81d9-57c110d05457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:59.693869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:59.697290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4d5b24a4-eec6-4d9a-81d9-57c110d05457 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:59.698491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4d5b24a4-eec6-4d9a-81d9-57c110d05457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:59.805842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:00.095214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21d9f3-1c54-4255-aa9b-1aca06349163 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:00.096813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8a21d9f3-1c54-4255-aa9b-1aca06349163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:00.210490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21d9f3-1c54-4255-aa9b-1aca06349163 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:00.264380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:00.291463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283223ca-fecc-4640-9a97-b5c8035d294e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:00.292078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283223ca-fecc-4640-9a97-b5c8035d294e service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:00.292352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283223ca-fecc-4640-9a97-b5c8035d294e service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:00.292623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283223ca-fecc-4640-9a97-b5c8035d294e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:00.294277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-283223ca-fecc-4640-9a97-b5c8035d294e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:00.413892 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283223ca-fecc-4640-9a97-b5c8035d294e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:00.415501 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283223ca-fecc-4640-9a97-b5c8035d294e service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:00.438837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 718/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:02.447289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a06bf-11bf-430b-b340-6e6315550a75 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:02.447877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a06bf-11bf-430b-b340-6e6315550a75 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:02.447877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a06bf-11bf-430b-b340-6e6315550a75 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:02.449261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:02.455714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d44cb-096b-4d8d-91fe-962dc86e4247 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:02.456411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d44cb-096b-4d8d-91fe-962dc86e4247 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:02.456506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d44cb-096b-4d8d-91fe-962dc86e4247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:02.545762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-db8d44cb-096b-4d8d-91fe-962dc86e4247 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:02.546970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-db8d44cb-096b-4d8d-91fe-962dc86e4247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:02.581130 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:02.658163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:02.947773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a06bf-11bf-430b-b340-6e6315550a75 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:02.949371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-338a06bf-11bf-430b-b340-6e6315550a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:03.063505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a06bf-11bf-430b-b340-6e6315550a75 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:03.098578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:03.147375 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00201e2-e3ce-4fbf-a6c7-df0d28032598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:03.147986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00201e2-e3ce-4fbf-a6c7-df0d28032598 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:03.148265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00201e2-e3ce-4fbf-a6c7-df0d28032598 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:03.148532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00201e2-e3ce-4fbf-a6c7-df0d28032598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:03.150184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d00201e2-e3ce-4fbf-a6c7-df0d28032598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:03.283077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00201e2-e3ce-4fbf-a6c7-df0d28032598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:03.284450 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00201e2-e3ce-4fbf-a6c7-df0d28032598 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:03.305830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 739/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:27:05.332256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fd5e4-2193-4cc3-b803-ec11db425dcb None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:05.332867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fd5e4-2193-4cc3-b803-ec11db425dcb None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:05.332867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fd5e4-2193-4cc3-b803-ec11db425dcb None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:05.334114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:05.340692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52343558-2f91-4f8d-a008-fec8e50e5e5a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:05.341410 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52343558-2f91-4f8d-a008-fec8e50e5e5a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:05.341593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52343558-2f91-4f8d-a008-fec8e50e5e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:05.420035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-52343558-2f91-4f8d-a008-fec8e50e5e5a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:05.421295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-52343558-2f91-4f8d-a008-fec8e50e5e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:05.430566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:05.531306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:05.815873 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fd5e4-2193-4cc3-b803-ec11db425dcb None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:05.818786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7b2fd5e4-2193-4cc3-b803-ec11db425dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:05.936829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fd5e4-2193-4cc3-b803-ec11db425dcb None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:05.998364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:06.019708 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615d5e8-aad6-4fbe-814f-1de4ed596a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:06.020323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615d5e8-aad6-4fbe-814f-1de4ed596a15 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:06.020603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615d5e8-aad6-4fbe-814f-1de4ed596a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:06.020903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615d5e8-aad6-4fbe-814f-1de4ed596a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:06.022534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5615d5e8-aad6-4fbe-814f-1de4ed596a15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:06.158736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615d5e8-aad6-4fbe-814f-1de4ed596a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:06.160324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615d5e8-aad6-4fbe-814f-1de4ed596a15 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:06.180050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 721/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:08.185953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca475a0a-6fb8-45a5-b6f0-ce73d926524d None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:08.186520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca475a0a-6fb8-45a5-b6f0-ce73d926524d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:08.186520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca475a0a-6fb8-45a5-b6f0-ce73d926524d None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:08.187955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:08.195037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbfdcb-22a6-4a7f-95d6-82bf2b6995d8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:08.195522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbfdcb-22a6-4a7f-95d6-82bf2b6995d8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:08.195662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbfdcb-22a6-4a7f-95d6-82bf2b6995d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:08.282480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9ccbfdcb-22a6-4a7f-95d6-82bf2b6995d8 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:08.283629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9ccbfdcb-22a6-4a7f-95d6-82bf2b6995d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:08.324527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:08.390469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:08.695700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca475a0a-6fb8-45a5-b6f0-ce73d926524d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:08.698441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ca475a0a-6fb8-45a5-b6f0-ce73d926524d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:08.815832 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca475a0a-6fb8-45a5-b6f0-ce73d926524d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:08.856863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:08.895120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be24b7-ef98-412d-8af8-2be1d06770d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:08.895731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be24b7-ef98-412d-8af8-2be1d06770d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:08.896000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be24b7-ef98-412d-8af8-2be1d06770d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:08.896263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be24b7-ef98-412d-8af8-2be1d06770d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:08.897899 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-83be24b7-ef98-412d-8af8-2be1d06770d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:09.014147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be24b7-ef98-412d-8af8-2be1d06770d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:09.015641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be24b7-ef98-412d-8af8-2be1d06770d6 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:09.036269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 742/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:10.989715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3758b-da2f-4a13-8c87-bfbda683c988 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:10.990512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3758b-da2f-4a13-8c87-bfbda683c988 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:10.990512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3758b-da2f-4a13-8c87-bfbda683c988 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:10.992811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:27:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:10.999372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276946a-62f0-4b86-ac10-db19272d9f95 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:11.000075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276946a-62f0-4b86-ac10-db19272d9f95 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:11.000251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276946a-62f0-4b86-ac10-db19272d9f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:11.072573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8276946a-62f0-4b86-ac10-db19272d9f95 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:11.073827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8276946a-62f0-4b86-ac10-db19272d9f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:11.095981 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:11.211086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:11.516678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3758b-da2f-4a13-8c87-bfbda683c988 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:11.519275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-18f3758b-da2f-4a13-8c87-bfbda683c988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:11.651498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3758b-da2f-4a13-8c87-bfbda683c988 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:11.743731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:11.752391 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3715898-824a-4f6b-bbc9-2e99b1290908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:11.753038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3715898-824a-4f6b-bbc9-2e99b1290908 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:11.753315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3715898-824a-4f6b-bbc9-2e99b1290908 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:11.753582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3715898-824a-4f6b-bbc9-2e99b1290908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:11.756045 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e3715898-824a-4f6b-bbc9-2e99b1290908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:11.896226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3715898-824a-4f6b-bbc9-2e99b1290908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:11.897746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3715898-824a-4f6b-bbc9-2e99b1290908 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:11.921296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 724/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:13.887205 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006691e-e546-49a6-ae20-a3c945709c76 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:13.888127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006691e-e546-49a6-ae20-a3c945709c76 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:13.888127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006691e-e546-49a6-ae20-a3c945709c76 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:13.891326 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:27:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:13.897490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425ff4e-abe8-4ff4-90ac-f626330c352a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:13.897963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425ff4e-abe8-4ff4-90ac-f626330c352a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:13.898067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425ff4e-abe8-4ff4-90ac-f626330c352a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:13.971418 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3425ff4e-abe8-4ff4-90ac-f626330c352a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:13.972897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3425ff4e-abe8-4ff4-90ac-f626330c352a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:14.026711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:14.086941 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:14.371295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006691e-e546-49a6-ae20-a3c945709c76 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:14.373874 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b006691e-e546-49a6-ae20-a3c945709c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:14.489739 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006691e-e546-49a6-ae20-a3c945709c76 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:14.538121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:14.565565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d08aac-474f-4aba-a063-b74c5604c58b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:14.566180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d08aac-474f-4aba-a063-b74c5604c58b service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:14.566457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d08aac-474f-4aba-a063-b74c5604c58b service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:14.566732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d08aac-474f-4aba-a063-b74c5604c58b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:14.568687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-34d08aac-474f-4aba-a063-b74c5604c58b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:14.697092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d08aac-474f-4aba-a063-b74c5604c58b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:14.698440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d08aac-474f-4aba-a063-b74c5604c58b service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:14.717301 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 745/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:24.084860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=22 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:24.714890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=66 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:34.261692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6440b0-16cf-4ed8-a4ce-98f24cd6194e None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:34.263235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6440b0-16cf-4ed8-a4ce-98f24cd6194e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:34.263235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6440b0-16cf-4ed8-a4ce-98f24cd6194e None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:34.264238 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 726/1471] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:27:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:34.273464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366083fc-2ac3-49c7-8746-ad408f03c9cd None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:34.274202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366083fc-2ac3-49c7-8746-ad408f03c9cd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:34.274444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366083fc-2ac3-49c7-8746-ad408f03c9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:34.350242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-366083fc-2ac3-49c7-8746-ad408f03c9cd None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:34.351460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-366083fc-2ac3-49c7-8746-ad408f03c9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:34.460124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 746/1472] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:34.754489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faddbb9-f6f0-4064-8a7b-200de9432a2a None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:34.754835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faddbb9-f6f0-4064-8a7b-200de9432a2a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:34.755193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faddbb9-f6f0-4064-8a7b-200de9432a2a None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:34.757343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 727/1473] 10.222.0.27 () {54 vars in 967 bytes} [Wed Aug 24 12:27:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:34.764532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd162d47-d423-43c9-ab2d-9a467535e1a3 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:34.764901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd162d47-d423-43c9-ab2d-9a467535e1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:34.765476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd162d47-d423-43c9-ab2d-9a467535e1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:34.850569 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-fd162d47-d423-43c9-ab2d-9a467535e1a3 None None] MFA Rules not processed for user `f94bfed415d24a9cb596b1fa92d5df97`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:34.851797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fd162d47-d423-43c9-ab2d-9a467535e1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:34.873771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:34.935895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 747/1474] 10.222.0.27 () {58 vars in 1064 bytes} [Wed Aug 24 12:27:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:34.945945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faddbb9-f6f0-4064-8a7b-200de9432a2a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:34.947530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3faddbb9-f6f0-4064-8a7b-200de9432a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:35.062558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faddbb9-f6f0-4064-8a7b-200de9432a2a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:35.142656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ac9ca-fc8d-4330-befe-86a5fbdceb06 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94bfed415d24a9cb596b1fa92d5df97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:35.143267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ac9ca-fc8d-4330-befe-86a5fbdceb06 service glance] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:35.143543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ac9ca-fc8d-4330-befe-86a5fbdceb06 service glance] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:35.143810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ac9ca-fc8d-4330-befe-86a5fbdceb06 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:35.145453 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4f1ac9ca-fc8d-4330-befe-86a5fbdceb06 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:35.205128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:35.267825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ac9ca-fc8d-4330-befe-86a5fbdceb06 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:35.269323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ac9ca-fc8d-4330-befe-86a5fbdceb06 service glance] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:35.288188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 728/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:37.278516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9ddac-3bb6-4426-b278-802170ab2671 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:37.279027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9ddac-3bb6-4426-b278-802170ab2671 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:37.279119 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9ddac-3bb6-4426-b278-802170ab2671 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:37.281242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 748/1476] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:27:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:37.289019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748d256-f3a1-4e29-b6ff-4c063ef5bd86 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:37.289511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748d256-f3a1-4e29-b6ff-4c063ef5bd86 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:37.289619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748d256-f3a1-4e29-b6ff-4c063ef5bd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:37.369963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7748d256-f3a1-4e29-b6ff-4c063ef5bd86 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:37.371904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7748d256-f3a1-4e29-b6ff-4c063ef5bd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:37.486209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 729/1477] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:37.755136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9ddac-3bb6-4426-b278-802170ab2671 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:37.758590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b4b9ddac-3bb6-4426-b278-802170ab2671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:37.917619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9ddac-3bb6-4426-b278-802170ab2671 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:37.985004 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:38.037599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617d73b-1a1b-45c7-9fd9-9aa57b7c793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:38.038116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617d73b-1a1b-45c7-9fd9-9aa57b7c793e admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:38.038395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617d73b-1a1b-45c7-9fd9-9aa57b7c793e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:38.038658 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617d73b-1a1b-45c7-9fd9-9aa57b7c793e admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:38.039302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617d73b-1a1b-45c7-9fd9-9aa57b7c793e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:38.041082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: WARNING py.warnings [None req-e617d73b-1a1b-45c7-9fd9-9aa57b7c793e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:27:38.041082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: warnings.warn(msg) Aug 24 12:27:38.041082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: Aug 24 12:27:38.042231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617d73b-1a1b-45c7-9fd9-9aa57b7c793e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:38.061656 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 749/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 12:27:37 2022] GET /identity/v3/projects => generated 2553 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:27:40.076282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a714f-fbb4-4e4c-82eb-6fe1206b1466 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:40.077007 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a714f-fbb4-4e4c-82eb-6fe1206b1466 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:40.077007 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a714f-fbb4-4e4c-82eb-6fe1206b1466 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:40.078307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 730/1479] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:40.086929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fc6aa-c45e-4bbd-856a-918ddae371a0 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:40.087426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fc6aa-c45e-4bbd-856a-918ddae371a0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:40.087426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fc6aa-c45e-4bbd-856a-918ddae371a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:40.171597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6e6fc6aa-c45e-4bbd-856a-918ddae371a0 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:40.173315 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6e6fc6aa-c45e-4bbd-856a-918ddae371a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:40.279985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 750/1480] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:40.586449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a714f-fbb4-4e4c-82eb-6fe1206b1466 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:40.588032 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-061a714f-fbb4-4e4c-82eb-6fe1206b1466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:40.598854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:40.746856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a714f-fbb4-4e4c-82eb-6fe1206b1466 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:40.841513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:40.884575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049d727-6944-4f37-b061-30720b8ab650 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:40.885208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049d727-6944-4f37-b061-30720b8ab650 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:40.885513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049d727-6944-4f37-b061-30720b8ab650 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:40.885752 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049d727-6944-4f37-b061-30720b8ab650 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:40.887399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3049d727-6944-4f37-b061-30720b8ab650 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:40.992957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049d727-6944-4f37-b061-30720b8ab650 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:40.994615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049d727-6944-4f37-b061-30720b8ab650 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:41.013287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 731/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:43.095788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a6d58-d3f5-4d14-b9c7-9514978b8561 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:43.096767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a6d58-d3f5-4d14-b9c7-9514978b8561 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:43.096767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a6d58-d3f5-4d14-b9c7-9514978b8561 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:43.099177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 751/1482] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:27:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:43.105341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ff7f5-e572-4253-a17a-9154483e0071 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:43.105817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ff7f5-e572-4253-a17a-9154483e0071 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:43.105919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ff7f5-e572-4253-a17a-9154483e0071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:43.187811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ba3ff7f5-e572-4253-a17a-9154483e0071 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:43.189080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ba3ff7f5-e572-4253-a17a-9154483e0071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:43.242540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:43.298186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 732/1483] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:43.596936 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a6d58-d3f5-4d14-b9c7-9514978b8561 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:43.599385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-546a6d58-d3f5-4d14-b9c7-9514978b8561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:43.632319 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:43.721447 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a6d58-d3f5-4d14-b9c7-9514978b8561 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:43.796988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a0e6d-ee49-4547-8573-f2dc7bda8d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:43.797609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a0e6d-ee49-4547-8573-f2dc7bda8d65 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:43.797887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a0e6d-ee49-4547-8573-f2dc7bda8d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:43.798153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a0e6d-ee49-4547-8573-f2dc7bda8d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:43.799766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-530a0e6d-ee49-4547-8573-f2dc7bda8d65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:43.890568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:43.927478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a0e6d-ee49-4547-8573-f2dc7bda8d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:43.929001 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a0e6d-ee49-4547-8573-f2dc7bda8d65 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:43.950684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 752/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:45.916461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8f76a-dbfe-4425-9e6e-7ebbca934ba6 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:45.917437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8f76a-dbfe-4425-9e6e-7ebbca934ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:45.917437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8f76a-dbfe-4425-9e6e-7ebbca934ba6 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:45.919648 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 733/1485] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:27:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:45.926143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99221309-db9b-4ab4-b37b-fa4f832460b6 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:45.926788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99221309-db9b-4ab4-b37b-fa4f832460b6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:45.926892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99221309-db9b-4ab4-b37b-fa4f832460b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:46.000892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-99221309-db9b-4ab4-b37b-fa4f832460b6 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:46.002111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-99221309-db9b-4ab4-b37b-fa4f832460b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:46.103986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 753/1486] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:46.404781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8f76a-dbfe-4425-9e6e-7ebbca934ba6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:46.406387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-97e8f76a-dbfe-4425-9e6e-7ebbca934ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:46.522707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8f76a-dbfe-4425-9e6e-7ebbca934ba6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:46.608296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:46.610277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5cf21-7a1d-4f43-8e50-97d71b371181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:46.611065 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5cf21-7a1d-4f43-8e50-97d71b371181 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:46.611358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5cf21-7a1d-4f43-8e50-97d71b371181 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:46.611635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5cf21-7a1d-4f43-8e50-97d71b371181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:46.614304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bca5cf21-7a1d-4f43-8e50-97d71b371181 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:46.734782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5cf21-7a1d-4f43-8e50-97d71b371181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:46.736281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5cf21-7a1d-4f43-8e50-97d71b371181 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:46.755657 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 734/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:48.750972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d42e64-82dc-4d71-9a6e-7eb1eee6248b None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:48.751900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d42e64-82dc-4d71-9a6e-7eb1eee6248b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:48.751900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d42e64-82dc-4d71-9a6e-7eb1eee6248b None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:48.754232 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 754/1488] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:27:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:48.760861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b632245-16b1-466f-a11e-5a12c5699b5a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:48.761284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b632245-16b1-466f-a11e-5a12c5699b5a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:48.761409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b632245-16b1-466f-a11e-5a12c5699b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:48.833405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3b632245-16b1-466f-a11e-5a12c5699b5a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:48.834691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3b632245-16b1-466f-a11e-5a12c5699b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:48.891376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:48.938623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 735/1489] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:49.235151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d42e64-82dc-4d71-9a6e-7eb1eee6248b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:49.237721 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-09d42e64-82dc-4d71-9a6e-7eb1eee6248b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:49.282207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:49.357724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d42e64-82dc-4d71-9a6e-7eb1eee6248b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:49.438780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed18854-c86e-4d24-aa5f-99f805faa2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:49.439390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed18854-c86e-4d24-aa5f-99f805faa2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:49.439667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed18854-c86e-4d24-aa5f-99f805faa2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:49.439933 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed18854-c86e-4d24-aa5f-99f805faa2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:49.441571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6ed18854-c86e-4d24-aa5f-99f805faa2e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:49.538611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:49.569407 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed18854-c86e-4d24-aa5f-99f805faa2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:49.571013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed18854-c86e-4d24-aa5f-99f805faa2e5 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:49.589869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 755/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:51.719108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803eb10-797a-4a8e-882f-7234a78cb8ff None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:51.719690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803eb10-797a-4a8e-882f-7234a78cb8ff None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:51.719690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803eb10-797a-4a8e-882f-7234a78cb8ff None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:51.721265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 736/1491] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 12:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:51.727530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86646ca8-525e-443b-a768-8d32dc42db30 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:51.728260 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86646ca8-525e-443b-a768-8d32dc42db30 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:51.728358 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86646ca8-525e-443b-a768-8d32dc42db30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:51.821264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-86646ca8-525e-443b-a768-8d32dc42db30 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:51.822486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-86646ca8-525e-443b-a768-8d32dc42db30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:51.928022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 756/1492] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:52.218466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803eb10-797a-4a8e-882f-7234a78cb8ff None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:52.220739 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a803eb10-797a-4a8e-882f-7234a78cb8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:52.341829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803eb10-797a-4a8e-882f-7234a78cb8ff None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:52.422833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a75b2c-6e93-466c-aa65-c498ffd2b14a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:52.423444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a75b2c-6e93-466c-aa65-c498ffd2b14a service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:52.423718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a75b2c-6e93-466c-aa65-c498ffd2b14a service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:52.423989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a75b2c-6e93-466c-aa65-c498ffd2b14a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:52.425639 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-99a75b2c-6e93-466c-aa65-c498ffd2b14a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:52.435567 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:52.563559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a75b2c-6e93-466c-aa65-c498ffd2b14a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:52.565057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a75b2c-6e93-466c-aa65-c498ffd2b14a service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:52.583928 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 737/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:28:01.925942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=82 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:28:02.581561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=50 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:05.153795 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b1ba4-ec0e-42a5-8270-e2019e65116e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:05.154728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b1ba4-ec0e-42a5-8270-e2019e65116e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:05.154728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b1ba4-ec0e-42a5-8270-e2019e65116e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:05.241692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-361b1ba4-ec0e-42a5-8270-e2019e65116e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:05.242849 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-361b1ba4-ec0e-42a5-8270-e2019e65116e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:05.341774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 757/1494] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:05.350169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64bd23-3f89-429f-848a-142df624b4e9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:05.350739 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64bd23-3f89-429f-848a-142df624b4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:05.350739 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64bd23-3f89-429f-848a-142df624b4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:05.433268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8d64bd23-3f89-429f-848a-142df624b4e9 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:05.434461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8d64bd23-3f89-429f-848a-142df624b4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:05.538933 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 738/1495] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:05.544557 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b1ba4-ec0e-42a5-8270-e2019e65116e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:05.546523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-361b1ba4-ec0e-42a5-8270-e2019e65116e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:05.562236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b1ba4-ec0e-42a5-8270-e2019e65116e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:05.631550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:05.668017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eb231-5b17-4df4-a212-d497c10b12c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:05.668625 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eb231-5b17-4df4-a212-d497c10b12c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:05.668933 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eb231-5b17-4df4-a212-d497c10b12c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:05.669188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eb231-5b17-4df4-a212-d497c10b12c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:05.669833 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823eb231-5b17-4df4-a212-d497c10b12c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:29:05.670132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823eb231-5b17-4df4-a212-d497c10b12c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:05.671162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: WARNING py.warnings [None req-823eb231-5b17-4df4-a212-d497c10b12c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:05.671162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: warnings.warn(msg) Aug 24 12:29:05.671162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: Aug 24 12:29:05.672059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823eb231-5b17-4df4-a212-d497c10b12c5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:05.684141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 758/1496] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:29:05.692876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64bd23-3f89-429f-848a-142df624b4e9 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:05.703148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64bd23-3f89-429f-848a-142df624b4e9 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:05.770480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:05.809733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9efbc-b54e-4a96-abad-d3e1351042ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:05.810438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9efbc-b54e-4a96-abad-d3e1351042ee admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:05.810779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9efbc-b54e-4a96-abad-d3e1351042ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:05.811124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9efbc-b54e-4a96-abad-d3e1351042ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:05.812035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9efbc-b54e-4a96-abad-d3e1351042ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:05.814812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-76e9efbc-b54e-4a96-abad-d3e1351042ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:05.814812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Aug 24 12:29:05.814812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: Aug 24 12:29:05.816614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9efbc-b54e-4a96-abad-d3e1351042ee admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:05.891930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 739/1497] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:05 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:05.897237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eb231-5b17-4df4-a212-d497c10b12c5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:05.899796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-823eb231-5b17-4df4-a212-d497c10b12c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:05.942276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eb231-5b17-4df4-a212-d497c10b12c5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:06.037919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d4ff8-167e-49c2-a9f2-a37d9ab05f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:06.038522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d4ff8-167e-49c2-a9f2-a37d9ab05f7d admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:06.038796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d4ff8-167e-49c2-a9f2-a37d9ab05f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:06.039060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d4ff8-167e-49c2-a9f2-a37d9ab05f7d admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:06.039649 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d4ff8-167e-49c2-a9f2-a37d9ab05f7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:06.040833 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: WARNING py.warnings [None req-cf8d4ff8-167e-49c2-a9f2-a37d9ab05f7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:06.040833 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: warnings.warn(msg) Aug 24 12:29:06.040833 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: Aug 24 12:29:06.042052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d4ff8-167e-49c2-a9f2-a37d9ab05f7d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:06.086918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 759/1498] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:05 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:06.091431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9efbc-b54e-4a96-abad-d3e1351042ee admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:06.093047 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-76e9efbc-b54e-4a96-abad-d3e1351042ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:06.160787 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9efbc-b54e-4a96-abad-d3e1351042ee admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:06.254923 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:06.266616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d2ed2-167c-4832-9656-dd13144ae145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:06.267256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d2ed2-167c-4832-9656-dd13144ae145 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:06.267534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d2ed2-167c-4832-9656-dd13144ae145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:06.267825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d2ed2-167c-4832-9656-dd13144ae145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:06.268476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d2ed2-167c-4832-9656-dd13144ae145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:06.269593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-ad2d2ed2-167c-4832-9656-dd13144ae145 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:06.269593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Aug 24 12:29:06.269593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: Aug 24 12:29:06.270482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d2ed2-167c-4832-9656-dd13144ae145 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:06.287868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 740/1499] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:06.292142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d4ff8-167e-49c2-a9f2-a37d9ab05f7d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:06.293723 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cf8d4ff8-167e-49c2-a9f2-a37d9ab05f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:06.305847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d4ff8-167e-49c2-a9f2-a37d9ab05f7d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:06.332057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:06.415522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acc1c3-3150-442d-99b2-30e541bffc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:06.416233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acc1c3-3150-442d-99b2-30e541bffc54 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:06.416513 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acc1c3-3150-442d-99b2-30e541bffc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:06.416826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acc1c3-3150-442d-99b2-30e541bffc54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:06.417469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71acc1c3-3150-442d-99b2-30e541bffc54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:06.418500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: WARNING py.warnings [None req-71acc1c3-3150-442d-99b2-30e541bffc54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:06.418500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: warnings.warn(msg) Aug 24 12:29:06.418500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: Aug 24 12:29:06.419403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71acc1c3-3150-442d-99b2-30e541bffc54 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:06.432323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 760/1500] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:06.437989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d2ed2-167c-4832-9656-dd13144ae145 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:06.439531 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ad2d2ed2-167c-4832-9656-dd13144ae145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:06.602861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d2ed2-167c-4832-9656-dd13144ae145 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:06.701106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca94052-002b-4f1c-832e-8fd7d3d806a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:06.701671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca94052-002b-4f1c-832e-8fd7d3d806a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:06.701945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca94052-002b-4f1c-832e-8fd7d3d806a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:06.702216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca94052-002b-4f1c-832e-8fd7d3d806a8 admin admin] PATH_INFO: `/v3/projects/0602af60f4f94b24af5b82d10f79add9/users/b2ea77b12a504c69a2629a1282429578/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:06.725437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca94052-002b-4f1c-832e-8fd7d3d806a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0602af60f4f94b24af5b82d10f79add9, user_id=b2ea77b12a504c69a2629a1282429578, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:06.726489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-5ca94052-002b-4f1c-832e-8fd7d3d806a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:06.726489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Aug 24 12:29:06.726489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: Aug 24 12:29:06.728548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca94052-002b-4f1c-832e-8fd7d3d806a8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:06.759341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 741/1501] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:06 2022] PUT /identity/v3/projects/0602af60f4f94b24af5b82d10f79add9/users/b2ea77b12a504c69a2629a1282429578/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:06.778044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86c05a-c553-4d7c-8024-f574182a1c2d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:06.778337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86c05a-c553-4d7c-8024-f574182a1c2d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:06.778609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86c05a-c553-4d7c-8024-f574182a1c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:06.840641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3c86c05a-c553-4d7c-8024-f574182a1c2d None None] MFA Rules not processed for user `b2ea77b12a504c69a2629a1282429578`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:06.841887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3c86c05a-c553-4d7c-8024-f574182a1c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:06.921852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 761/1502] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:29:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:06.935509 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29a486-59d4-4f61-b90d-1c22ebd9e66f None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:06.935796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29a486-59d4-4f61-b90d-1c22ebd9e66f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:06.936070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29a486-59d4-4f61-b90d-1c22ebd9e66f None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:06.937469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 742/1503] 10.222.0.27 () {54 vars in 964 bytes} [Wed Aug 24 12:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:29:06.942469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504bc30-02d5-43fe-b1fe-cff552d5b7f7 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:06.942750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504bc30-02d5-43fe-b1fe-cff552d5b7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:06.943021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504bc30-02d5-43fe-b1fe-cff552d5b7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:06.979894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:07.029085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7504bc30-02d5-43fe-b1fe-cff552d5b7f7 None None] MFA Rules not processed for user `a4b53c9f4ff842e08ec41f53147d0431`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:07.030283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7504bc30-02d5-43fe-b1fe-cff552d5b7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:07.116221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 762/1504] 10.222.0.27 () {58 vars in 1061 bytes} [Wed Aug 24 12:29:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:07.124336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29a486-59d4-4f61-b90d-1c22ebd9e66f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:07.125899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1c29a486-59d4-4f61-b90d-1c22ebd9e66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:07.177861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:07.235943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29a486-59d4-4f61-b90d-1c22ebd9e66f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:07.315950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80099a9d-bf02-452d-ab4c-7dfce7d49b11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:07.316545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80099a9d-bf02-452d-ab4c-7dfce7d49b11 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:07.316838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80099a9d-bf02-452d-ab4c-7dfce7d49b11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:07.317109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80099a9d-bf02-452d-ab4c-7dfce7d49b11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:07.318691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-80099a9d-bf02-452d-ab4c-7dfce7d49b11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:07.421994 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80099a9d-bf02-452d-ab4c-7dfce7d49b11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:07.423438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80099a9d-bf02-452d-ab4c-7dfce7d49b11 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:07.426979 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:07.442874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 743/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:07.494968 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6005f6f-a35e-4fb4-bbbe-c87a23a7e980 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:07.495442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6005f6f-a35e-4fb4-bbbe-c87a23a7e980 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:07.495543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6005f6f-a35e-4fb4-bbbe-c87a23a7e980 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:07.496846 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 763/1506] 10.222.0.27 () {52 vars in 830 bytes} [Wed Aug 24 12:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:29:07.500689 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80099a9d-bf02-452d-ab4c-7dfce7d49b11 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:07.501932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-80099a9d-bf02-452d-ab4c-7dfce7d49b11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:07.623433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80099a9d-bf02-452d-ab4c-7dfce7d49b11 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:07.729145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:07.729772 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:07.730049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:07.730306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] PATH_INFO: `/v3/users/b2ea77b12a504c69a2629a1282429578` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:07.744605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ea77b12a504c69a2629a1282429578)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:07.745615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:07.745615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Aug 24 12:29:07.745615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: Aug 24 12:29:07.746810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:07.761253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:07.829897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ea77b12a504c69a2629a1282429578'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:07.830336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ea77b12a504c69a2629a1282429578'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:07.830864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ea77b12a504c69a2629a1282429578'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:07.855816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ea77b12a504c69a2629a1282429578'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:07.868709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ea77b12a504c69a2629a1282429578'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:07.894554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 744/1507] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:29:07 2022] DELETE /identity/v3/users/b2ea77b12a504c69a2629a1282429578 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:07.901052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022fa0f-67db-4029-b1ae-86e96609f7ef None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:07.901353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022fa0f-67db-4029-b1ae-86e96609f7ef None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:07.901631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022fa0f-67db-4029-b1ae-86e96609f7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:07.963488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5022fa0f-67db-4029-b1ae-86e96609f7ef None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:07.964736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5022fa0f-67db-4029-b1ae-86e96609f7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:08.051989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:08.067364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 764/1508] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:08.080827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:08.082163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:08.190128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cadf7e-af9c-48f1-bbd2-2e408d508dc3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:08.255585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:08.282735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7f0e4-ac8a-433b-8fb9-b2a96f28e952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:08.283369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7f0e4-ac8a-433b-8fb9-b2a96f28e952 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:08.283646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7f0e4-ac8a-433b-8fb9-b2a96f28e952 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:08.283915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7f0e4-ac8a-433b-8fb9-b2a96f28e952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:08.285563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0ad7f0e4-ac8a-433b-8fb9-b2a96f28e952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:08.402571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7f0e4-ac8a-433b-8fb9-b2a96f28e952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:08.404022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7f0e4-ac8a-433b-8fb9-b2a96f28e952 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:08.422823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 745/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:08.862288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022fa0f-67db-4029-b1ae-86e96609f7ef None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:08.863530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5022fa0f-67db-4029-b1ae-86e96609f7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:08.875158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022fa0f-67db-4029-b1ae-86e96609f7ef None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:08.975126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:08.975672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:08.975945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:08.976223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] PATH_INFO: `/v3/projects/0602af60f4f94b24af5b82d10f79add9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:08.988530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0602af60f4f94b24af5b82d10f79add9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:08.989635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: WARNING py.warnings [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:08.989635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: warnings.warn(msg) Aug 24 12:29:08.989635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: Aug 24 12:29:08.990854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:09.067402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0602af60f4f94b24af5b82d10f79add9', 'request_id': 'req-25a58d8e-27b0-4a25-ac48-29ed85831c40'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:09.079190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0602af60f4f94b24af5b82d10f79add9', 'request_id': 'req-25a58d8e-27b0-4a25-ac48-29ed85831c40'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:09.093228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0602af60f4f94b24af5b82d10f79add9', 'request_id': 'req-25a58d8e-27b0-4a25-ac48-29ed85831c40'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:09.118760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] The token cache is being invalidate because project 0602af60f4f94b24af5b82d10f79add9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:29:09.119039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:09.119560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-25a58d8e-27b0-4a25-ac48-29ed85831c40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:09.120933 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 765/1510] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:29:08 2022] DELETE /identity/v3/projects/0602af60f4f94b24af5b82d10f79add9 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:18.420484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=62 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:19.116204 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:42.355919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2ce05-f514-423e-b63d-e9c7cb18de5c None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:42.356539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2ce05-f514-423e-b63d-e9c7cb18de5c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:42.356539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2ce05-f514-423e-b63d-e9c7cb18de5c None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:42.357838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:42.363143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640c507-489f-4bfd-80d6-ec152536bb71 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:42.363650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640c507-489f-4bfd-80d6-ec152536bb71 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:42.363810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640c507-489f-4bfd-80d6-ec152536bb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:42.455624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1640c507-489f-4bfd-80d6-ec152536bb71 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:42.456971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1640c507-489f-4bfd-80d6-ec152536bb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:42.570498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:42.735750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2ce05-f514-423e-b63d-e9c7cb18de5c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:42.738221 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-97c2ce05-f514-423e-b63d-e9c7cb18de5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:42.863773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2ce05-f514-423e-b63d-e9c7cb18de5c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:42.870866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125822aa-503d-47b4-9869-81580dc9ff81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:42.871475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125822aa-503d-47b4-9869-81580dc9ff81 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:42.871753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125822aa-503d-47b4-9869-81580dc9ff81 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:42.872023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125822aa-503d-47b4-9869-81580dc9ff81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:42.883146 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125822aa-503d-47b4-9869-81580dc9ff81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:42.884796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125822aa-503d-47b4-9869-81580dc9ff81 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:42.901984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 747/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:43.771503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4988dbd-8cc1-4b21-9f97-e8fef8b6bc68 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:43.772020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4988dbd-8cc1-4b21-9f97-e8fef8b6bc68 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:43.772020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4988dbd-8cc1-4b21-9f97-e8fef8b6bc68 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:43.773468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:43.779419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df917db3-2cc1-40d4-937d-66c4fd1c2626 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:43.779876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df917db3-2cc1-40d4-937d-66c4fd1c2626 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:43.779983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df917db3-2cc1-40d4-937d-66c4fd1c2626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:43.807365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-df917db3-2cc1-40d4-937d-66c4fd1c2626 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:43.808585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-df917db3-2cc1-40d4-937d-66c4fd1c2626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:43.817604 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:43.987761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4988dbd-8cc1-4b21-9f97-e8fef8b6bc68 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:44.003991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4988dbd-8cc1-4b21-9f97-e8fef8b6bc68 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:44.009851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0503e-ed29-41e7-9feb-7a63bed8ea9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:44.010470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0503e-ed29-41e7-9feb-7a63bed8ea9b service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:44.010750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0503e-ed29-41e7-9feb-7a63bed8ea9b service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:44.011019 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0503e-ed29-41e7-9feb-7a63bed8ea9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:44.019659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0503e-ed29-41e7-9feb-7a63bed8ea9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:44.021308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0503e-ed29-41e7-9feb-7a63bed8ea9b service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:44.031249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 768/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:44.189452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e3601-c5f5-4bb8-b876-77bec12c9d44 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:44.189710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e3601-c5f5-4bb8-b876-77bec12c9d44 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:44.189986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e3601-c5f5-4bb8-b876-77bec12c9d44 None None] PATH_INFO: `/v3` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:44.191276 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 749/1517] 10.222.0.55 () {54 vars in 925 bytes} [Wed Aug 24 12:30:44 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:30:44.201461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08dc4b-9dc0-4801-a1a5-45af42635dd9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:44.201748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08dc4b-9dc0-4801-a1a5-45af42635dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:44.202072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08dc4b-9dc0-4801-a1a5-45af42635dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:44.274041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ba08dc4b-9dc0-4801-a1a5-45af42635dd9 None None] MFA Rules not processed for user `6db648aaa06b476e9a3cafbab832ee68`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:44.275873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ba08dc4b-9dc0-4801-a1a5-45af42635dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:44.297110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:44.356684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 769/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Aug 24 12:30:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:30:53.771924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5424977-cb51-4745-9658-90aec21e75d7 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:53.772534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5424977-cb51-4745-9658-90aec21e75d7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:53.772534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5424977-cb51-4745-9658-90aec21e75d7 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:53.774622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 750/1519] 10.222.0.27 () {54 vars in 961 bytes} [Wed Aug 24 12:30:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:53.779874 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf30dc9-6743-4247-98af-fe7d58e8c7ce None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:53.780330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf30dc9-6743-4247-98af-fe7d58e8c7ce None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:53.780447 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf30dc9-6743-4247-98af-fe7d58e8c7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:53.800568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-fcf30dc9-6743-4247-98af-fe7d58e8c7ce None None] MFA Rules not processed for user `6db648aaa06b476e9a3cafbab832ee68`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:53.802992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fcf30dc9-6743-4247-98af-fe7d58e8c7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:53.803164 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=56 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:53.813363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 770/1520] 10.222.0.27 () {58 vars in 1058 bytes} [Wed Aug 24 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:53.821994 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5424977-cb51-4745-9658-90aec21e75d7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:53.842760 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5424977-cb51-4745-9658-90aec21e75d7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:53.848741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920877e8-e335-4a7d-a2d4-326baf543b48 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:53.849323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920877e8-e335-4a7d-a2d4-326baf543b48 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:53.849598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920877e8-e335-4a7d-a2d4-326baf543b48 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:53.849858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920877e8-e335-4a7d-a2d4-326baf543b48 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:53.858215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920877e8-e335-4a7d-a2d4-326baf543b48 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:53.859802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920877e8-e335-4a7d-a2d4-326baf543b48 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:53.869502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 751/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:30:55.308878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5aba8-29be-4e69-9399-300226d04092 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:55.309440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5aba8-29be-4e69-9399-300226d04092 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:55.309440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5aba8-29be-4e69-9399-300226d04092 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:55.310708 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:55.315964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9d0aa-4152-461d-a481-505ab26bed11 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:55.316412 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9d0aa-4152-461d-a481-505ab26bed11 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:55.316516 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9d0aa-4152-461d-a481-505ab26bed11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:55.338480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-16f9d0aa-4152-461d-a481-505ab26bed11 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:55.339676 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-16f9d0aa-4152-461d-a481-505ab26bed11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:55.348748 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:55.505978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5aba8-29be-4e69-9399-300226d04092 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:55.529383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5aba8-29be-4e69-9399-300226d04092 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:55.537065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75075984-f7d6-4115-b4c2-7de2b2dbeca6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:55.537664 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075984-f7d6-4115-b4c2-7de2b2dbeca6 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:55.537943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075984-f7d6-4115-b4c2-7de2b2dbeca6 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:55.538213 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075984-f7d6-4115-b4c2-7de2b2dbeca6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:55.547493 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75075984-f7d6-4115-b4c2-7de2b2dbeca6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:55.549249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75075984-f7d6-4115-b4c2-7de2b2dbeca6 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:55.559822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 772/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:56.410514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a3ecf-71b7-413f-89b8-6b4fd09fd638 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.411089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a3ecf-71b7-413f-89b8-6b4fd09fd638 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.411089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a3ecf-71b7-413f-89b8-6b4fd09fd638 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.412427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:56.418468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9d3fa-e31f-42c5-bf16-13805ecc810a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.419198 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9d3fa-e31f-42c5-bf16-13805ecc810a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.419351 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9d3fa-e31f-42c5-bf16-13805ecc810a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.441178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-77f9d3fa-e31f-42c5-bf16-13805ecc810a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:56.442397 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-77f9d3fa-e31f-42c5-bf16-13805ecc810a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:56.451583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:56.607559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a3ecf-71b7-413f-89b8-6b4fd09fd638 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:56.625574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a3ecf-71b7-413f-89b8-6b4fd09fd638 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:56.631837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf489c-362c-4a2d-9e0c-d2db6206e600 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:56.632430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf489c-362c-4a2d-9e0c-d2db6206e600 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.632738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf489c-362c-4a2d-9e0c-d2db6206e600 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.633012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf489c-362c-4a2d-9e0c-d2db6206e600 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.642062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf489c-362c-4a2d-9e0c-d2db6206e600 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:56.643671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf489c-362c-4a2d-9e0c-d2db6206e600 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:56.653051 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 754/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:00.600482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c56a6c-133a-411b-a32a-07b3ae04e822 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:00.601501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c56a6c-133a-411b-a32a-07b3ae04e822 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:00.601501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c56a6c-133a-411b-a32a-07b3ae04e822 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:00.603570 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:31:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:31:00.612310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a7aa9-7248-4264-8314-2281bcbd1906 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:00.612782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a7aa9-7248-4264-8314-2281bcbd1906 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:00.612875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a7aa9-7248-4264-8314-2281bcbd1906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:00.638620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ec7a7aa9-7248-4264-8314-2281bcbd1906 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:00.639830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ec7a7aa9-7248-4264-8314-2281bcbd1906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:00.649830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:31:00.940752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c56a6c-133a-411b-a32a-07b3ae04e822 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:00.943954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-93c56a6c-133a-411b-a32a-07b3ae04e822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:01.034791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c56a6c-133a-411b-a32a-07b3ae04e822 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:01.038949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144c38b-1251-450f-9117-f68aaf58b9a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:01.039560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144c38b-1251-450f-9117-f68aaf58b9a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:01.039834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144c38b-1251-450f-9117-f68aaf58b9a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:01.040097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144c38b-1251-450f-9117-f68aaf58b9a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:01.050957 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144c38b-1251-450f-9117-f68aaf58b9a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:01.052389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144c38b-1251-450f-9117-f68aaf58b9a2 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:01.062623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 775/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:02.967402 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f36fd-e389-486c-946f-23093cf346f8 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:02.967963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f36fd-e389-486c-946f-23093cf346f8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:02.967963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f36fd-e389-486c-946f-23093cf346f8 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:02.969442 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:31:02.977853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d80e4-0269-439a-b07a-bdca619222b4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:02.978396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d80e4-0269-439a-b07a-bdca619222b4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:02.978493 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d80e4-0269-439a-b07a-bdca619222b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:03.010287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-247d80e4-0269-439a-b07a-bdca619222b4 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:03.011516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-247d80e4-0269-439a-b07a-bdca619222b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:03.021443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:03.309771 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f36fd-e389-486c-946f-23093cf346f8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:03.329726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f36fd-e389-486c-946f-23093cf346f8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:03.334324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0faf7a-89ce-4425-add8-bed7bd6608cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:03.334914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0faf7a-89ce-4425-add8-bed7bd6608cf service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:03.335156 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0faf7a-89ce-4425-add8-bed7bd6608cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:03.335459 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0faf7a-89ce-4425-add8-bed7bd6608cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:03.345621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0faf7a-89ce-4425-add8-bed7bd6608cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:03.347066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0faf7a-89ce-4425-add8-bed7bd6608cf service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:03.356857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 757/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:06.894484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab0f58-9172-42a3-9356-641aa0d08378 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:06.895094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab0f58-9172-42a3-9356-641aa0d08378 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:06.895094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab0f58-9172-42a3-9356-641aa0d08378 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:06.897201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:31:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:31:06.906123 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77d43b-6ef7-4d30-abc2-8d60eb46b34a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:06.906571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77d43b-6ef7-4d30-abc2-8d60eb46b34a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:06.907322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77d43b-6ef7-4d30-abc2-8d60eb46b34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:06.928850 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7a77d43b-6ef7-4d30-abc2-8d60eb46b34a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:06.930073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7a77d43b-6ef7-4d30-abc2-8d60eb46b34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:06.940000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:07.511856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab0f58-9172-42a3-9356-641aa0d08378 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:07.514845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5bab0f58-9172-42a3-9356-641aa0d08378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:07.602408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:07.617405 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab0f58-9172-42a3-9356-641aa0d08378 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:07.621456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e1ffe-3a07-4cd7-8bdd-fc5483d18787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:07.622066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e1ffe-3a07-4cd7-8bdd-fc5483d18787 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:07.622346 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e1ffe-3a07-4cd7-8bdd-fc5483d18787 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:07.622615 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e1ffe-3a07-4cd7-8bdd-fc5483d18787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:07.632221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e1ffe-3a07-4cd7-8bdd-fc5483d18787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:07.633692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e1ffe-3a07-4cd7-8bdd-fc5483d18787 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:07.643354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 778/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:09.558445 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19ca6c-cd55-46e1-bd57-6d8700614fef None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:09.559375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19ca6c-cd55-46e1-bd57-6d8700614fef None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:09.559375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19ca6c-cd55-46e1-bd57-6d8700614fef None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:09.561557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:31:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:31:09.567991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c26c2-3d78-4946-9650-6b8959fe5f25 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:09.568473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c26c2-3d78-4946-9650-6b8959fe5f25 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:09.568579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c26c2-3d78-4946-9650-6b8959fe5f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:09.596468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2a5c26c2-3d78-4946-9650-6b8959fe5f25 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:31:09.597715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2a5c26c2-3d78-4946-9650-6b8959fe5f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:09.606767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:31:10.152380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19ca6c-cd55-46e1-bd57-6d8700614fef None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:31:10.154371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0a19ca6c-cd55-46e1-bd57-6d8700614fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:31:10.171458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19ca6c-cd55-46e1-bd57-6d8700614fef None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:31:10.176306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d870383-0f70-47b4-875d-698c76281630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:31:10.176967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d870383-0f70-47b4-875d-698c76281630 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:31:10.177243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d870383-0f70-47b4-875d-698c76281630 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:31:10.177511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d870383-0f70-47b4-875d-698c76281630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:31:10.188105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d870383-0f70-47b4-875d-698c76281630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:31:10.189856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d870383-0f70-47b4-875d-698c76281630 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:31:10.200437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 760/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:31:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:19.589967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=11 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:20.196007 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=33 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:34.234970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a9100-46a5-4e97-980a-b0b586af94a6 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.235937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a9100-46a5-4e97-980a-b0b586af94a6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.236384 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a9100-46a5-4e97-980a-b0b586af94a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.266934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0a5a9100-46a5-4e97-980a-b0b586af94a6 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.268498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0a5a9100-46a5-4e97-980a-b0b586af94a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.282634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 780/1540] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:34.291849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2f0d6-e762-4504-9f21-895fa1b2405c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.297152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2f0d6-e762-4504-9f21-895fa1b2405c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.297152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2f0d6-e762-4504-9f21-895fa1b2405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.329298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1fe2f0d6-e762-4504-9f21-895fa1b2405c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.330588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1fe2f0d6-e762-4504-9f21-895fa1b2405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.343272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 761/1541] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:34.348827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a9100-46a5-4e97-980a-b0b586af94a6 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.359742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a9100-46a5-4e97-980a-b0b586af94a6 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.361211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5660d92-238d-4db5-98c6-7d1993422d4a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.361501 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5660d92-238d-4db5-98c6-7d1993422d4a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.361775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5660d92-238d-4db5-98c6-7d1993422d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.371293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fadff-7b88-4fab-84a7-3646559e2c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.372480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fadff-7b88-4fab-84a7-3646559e2c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.373003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fadff-7b88-4fab-84a7-3646559e2c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.373506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fadff-7b88-4fab-84a7-3646559e2c60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.374384 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fadff-7b88-4fab-84a7-3646559e2c60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:34.374903 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fadff-7b88-4fab-84a7-3646559e2c60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.376766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fadff-7b88-4fab-84a7-3646559e2c60 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.395127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 781/1542] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:32:34.403503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b5660d92-238d-4db5-98c6-7d1993422d4a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.404791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b5660d92-238d-4db5-98c6-7d1993422d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.414252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fadff-7b88-4fab-84a7-3646559e2c60 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.420403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 762/1543] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:34.430140 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f13df4-7701-4c02-9bed-9989240b7a01 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.430464 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f13df4-7701-4c02-9bed-9989240b7a01 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.430692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f13df4-7701-4c02-9bed-9989240b7a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.431122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fadff-7b88-4fab-84a7-3646559e2c60 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.453427 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969172c2-cf58-48d5-96c3-00161546e32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.454468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969172c2-cf58-48d5-96c3-00161546e32a admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.454947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969172c2-cf58-48d5-96c3-00161546e32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.455408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969172c2-cf58-48d5-96c3-00161546e32a admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.456298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969172c2-cf58-48d5-96c3-00161546e32a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.457702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: WARNING py.warnings [None req-969172c2-cf58-48d5-96c3-00161546e32a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:32:34.457702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: warnings.warn(msg) Aug 24 12:32:34.457702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: Aug 24 12:32:34.459454 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969172c2-cf58-48d5-96c3-00161546e32a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.466816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b2f13df4-7701-4c02-9bed-9989240b7a01 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.468057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b2f13df4-7701-4c02-9bed-9989240b7a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.491439 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 763/1544] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:34.493159 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1953dd1-558b-46d4-ab09-10ce8b62498a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.493159 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1953dd1-558b-46d4-ab09-10ce8b62498a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.493509 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1953dd1-558b-46d4-ab09-10ce8b62498a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.523207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b1953dd1-558b-46d4-ab09-10ce8b62498a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.524571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b1953dd1-558b-46d4-ab09-10ce8b62498a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.537908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 782/1545] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:34.539145 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969172c2-cf58-48d5-96c3-00161546e32a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.543566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 764/1546] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:34.546210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1953dd1-558b-46d4-ab09-10ce8b62498a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.562058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1953dd1-558b-46d4-ab09-10ce8b62498a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.614535 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969172c2-cf58-48d5-96c3-00161546e32a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.629989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0d6f8-17d7-4070-94a9-48b0630a794a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.630927 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0d6f8-17d7-4070-94a9-48b0630a794a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.631485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0d6f8-17d7-4070-94a9-48b0630a794a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.631949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0d6f8-17d7-4070-94a9-48b0630a794a admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.632834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0d6f8-17d7-4070-94a9-48b0630a794a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:34.633443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0d6f8-17d7-4070-94a9-48b0630a794a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.635341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0d6f8-17d7-4070-94a9-48b0630a794a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.648960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040ddd7-4e57-4cda-be5e-4aa30aa6a0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.649950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040ddd7-4e57-4cda-be5e-4aa30aa6a0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.650496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040ddd7-4e57-4cda-be5e-4aa30aa6a0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.650957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040ddd7-4e57-4cda-be5e-4aa30aa6a0b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.651784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040ddd7-4e57-4cda-be5e-4aa30aa6a0b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.653136 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-b040ddd7-4e57-4cda-be5e-4aa30aa6a0b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:32:34.653136 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Aug 24 12:32:34.653136 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: Aug 24 12:32:34.655184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040ddd7-4e57-4cda-be5e-4aa30aa6a0b9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.664676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 783/1547] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:34.664676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc1f83-a0f4-45dd-9dee-88056be75ea8 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.664676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc1f83-a0f4-45dd-9dee-88056be75ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.664676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc1f83-a0f4-45dd-9dee-88056be75ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.698934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 765/1548] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:34.700074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3efc1f83-a0f4-45dd-9dee-88056be75ea8 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.700227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd5d1d-9015-4cde-b8dc-b5b844af83c6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.700409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd5d1d-9015-4cde-b8dc-b5b844af83c6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.702106 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3efc1f83-a0f4-45dd-9dee-88056be75ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.703633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd5d1d-9015-4cde-b8dc-b5b844af83c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.715299 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 784/1549] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:34.717473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc80d4-2882-406d-b09d-0aab8d055a1e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.717924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc80d4-2882-406d-b09d-0aab8d055a1e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.718427 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc80d4-2882-406d-b09d-0aab8d055a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.732669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-14bd5d1d-9015-4cde-b8dc-b5b844af83c6 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.733898 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-14bd5d1d-9015-4cde-b8dc-b5b844af83c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.741263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-34cc80d4-2882-406d-b09d-0aab8d055a1e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.742496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-34cc80d4-2882-406d-b09d-0aab8d055a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.744447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 766/1550] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:34.744447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7f2ed-edd3-435c-8ebc-5a63e2f04655 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.744779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7f2ed-edd3-435c-8ebc-5a63e2f04655 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.745042 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7f2ed-edd3-435c-8ebc-5a63e2f04655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.755784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 785/1551] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:34.756538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc80d4-2882-406d-b09d-0aab8d055a1e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.765804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2fd7f2ed-edd3-435c-8ebc-5a63e2f04655 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.767034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2fd7f2ed-edd3-435c-8ebc-5a63e2f04655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.775195 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 767/1552] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:34.775849 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc80d4-2882-406d-b09d-0aab8d055a1e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.776036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7f2ed-edd3-435c-8ebc-5a63e2f04655 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.781825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34abe6a-88cb-404b-8ec5-c64762c643f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.782746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34abe6a-88cb-404b-8ec5-c64762c643f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.783258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34abe6a-88cb-404b-8ec5-c64762c643f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.783528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34abe6a-88cb-404b-8ec5-c64762c643f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.784358 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34abe6a-88cb-404b-8ec5-c64762c643f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.786100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7f2ed-edd3-435c-8ebc-5a63e2f04655 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.786352 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34abe6a-88cb-404b-8ec5-c64762c643f5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.790997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47abdb-66b7-4ae8-8d0f-05a06f9c5b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.791605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47abdb-66b7-4ae8-8d0f-05a06f9c5b52 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.791881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47abdb-66b7-4ae8-8d0f-05a06f9c5b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.792150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47abdb-66b7-4ae8-8d0f-05a06f9c5b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.792835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47abdb-66b7-4ae8-8d0f-05a06f9c5b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.794457 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47abdb-66b7-4ae8-8d0f-05a06f9c5b52 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.805006 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 768/1553] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:34.805904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47abdb-66b7-4ae8-8d0f-05a06f9c5b52 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.821576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 786/1554] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:34.821869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47abdb-66b7-4ae8-8d0f-05a06f9c5b52 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.821978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d1f1b-20e3-4581-b3c3-33557d84d1b6 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.821978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d1f1b-20e3-4581-b3c3-33557d84d1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.821978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d1f1b-20e3-4581-b3c3-33557d84d1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.839297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7e2d1f1b-20e3-4581-b3c3-33557d84d1b6 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.840488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7e2d1f1b-20e3-4581-b3c3-33557d84d1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.877319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0592e-8e1f-460f-846c-f78b3a112dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.878036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0592e-8e1f-460f-846c-f78b3a112dfa admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.878236 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0592e-8e1f-460f-846c-f78b3a112dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.878504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0592e-8e1f-460f-846c-f78b3a112dfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.879176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0592e-8e1f-460f-846c-f78b3a112dfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:34.879490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0592e-8e1f-460f-846c-f78b3a112dfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.880507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-51a0592e-8e1f-460f-846c-f78b3a112dfa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:32:34.880507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Aug 24 12:32:34.880507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: Aug 24 12:32:34.881433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0592e-8e1f-460f-846c-f78b3a112dfa admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.893390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 769/1555] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:34.894880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc7053-ea28-4564-a834-b3c576a556d9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.895157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc7053-ea28-4564-a834-b3c576a556d9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.895423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc7053-ea28-4564-a834-b3c576a556d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.910283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 787/1556] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:34.912039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c12508-3f2e-4c73-a6ca-f317960a0b68 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.912323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c12508-3f2e-4c73-a6ca-f317960a0b68 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.912589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c12508-3f2e-4c73-a6ca-f317960a0b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.919702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-64cc7053-ea28-4564-a834-b3c576a556d9 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.921139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-64cc7053-ea28-4564-a834-b3c576a556d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.929618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 770/1557] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:34.930996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa294f-030c-4d34-981c-4653bf7e9142 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.931295 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa294f-030c-4d34-981c-4653bf7e9142 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.931567 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa294f-030c-4d34-981c-4653bf7e9142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.934305 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-51c12508-3f2e-4c73-a6ca-f317960a0b68 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.935486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-51c12508-3f2e-4c73-a6ca-f317960a0b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.944075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 788/1558] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:34.945341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802dd92-e692-400b-ad3c-fca2eaf16dfe None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.945619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802dd92-e692-400b-ad3c-fca2eaf16dfe None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.945898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802dd92-e692-400b-ad3c-fca2eaf16dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.953769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:34.953949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-53aa294f-030c-4d34-981c-4653bf7e9142 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.955149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-53aa294f-030c-4d34-981c-4653bf7e9142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.963258 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 771/1559] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:34.964108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa294f-030c-4d34-981c-4653bf7e9142 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.967840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1802dd92-e692-400b-ad3c-fca2eaf16dfe None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:34.969170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1802dd92-e692-400b-ad3c-fca2eaf16dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.973869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa294f-030c-4d34-981c-4653bf7e9142 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.984190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 789/1560] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:34.984342 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c45fdc-c1e0-444e-bc70-30bbe76fe2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.985109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c45fdc-c1e0-444e-bc70-30bbe76fe2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.985400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802dd92-e692-400b-ad3c-fca2eaf16dfe None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.985521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c45fdc-c1e0-444e-bc70-30bbe76fe2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.985811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c45fdc-c1e0-444e-bc70-30bbe76fe2f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.986522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c45fdc-c1e0-444e-bc70-30bbe76fe2f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.988493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c45fdc-c1e0-444e-bc70-30bbe76fe2f6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.997352 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802dd92-e692-400b-ad3c-fca2eaf16dfe None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.002263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 772/1561] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:35.002263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c45fdc-c1e0-444e-bc70-30bbe76fe2f6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.006390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272d36c-b1ea-4a43-b914-c777c02f5973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.007078 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272d36c-b1ea-4a43-b914-c777c02f5973 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.007357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272d36c-b1ea-4a43-b914-c777c02f5973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.007626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272d36c-b1ea-4a43-b914-c777c02f5973 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.008272 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272d36c-b1ea-4a43-b914-c777c02f5973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.009842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c45fdc-c1e0-444e-bc70-30bbe76fe2f6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.010297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272d36c-b1ea-4a43-b914-c777c02f5973 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.014489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282312d-b142-4dda-b41b-e92b7d39c19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.015058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282312d-b142-4dda-b41b-e92b7d39c19e admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.015333 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282312d-b142-4dda-b41b-e92b7d39c19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.015596 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282312d-b142-4dda-b41b-e92b7d39c19e admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.016232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282312d-b142-4dda-b41b-e92b7d39c19e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.018113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282312d-b142-4dda-b41b-e92b7d39c19e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.024433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:35.039769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 790/1562] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/users => generated 293 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:35.039769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc06170-c68f-4696-8ead-68579daa24ab None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.039769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc06170-c68f-4696-8ead-68579daa24ab None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.039769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc06170-c68f-4696-8ead-68579daa24ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.055243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 773/1563] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:34 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:35.055243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282312d-b142-4dda-b41b-e92b7d39c19e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.058596 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4bc06170-c68f-4696-8ead-68579daa24ab None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:35.059761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4bc06170-c68f-4696-8ead-68579daa24ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:35.118442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 791/1564] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:35.119449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc06170-c68f-4696-8ead-68579daa24ab None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.124151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282312d-b142-4dda-b41b-e92b7d39c19e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.129951 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc06170-c68f-4696-8ead-68579daa24ab None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.130604 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb082f-5d69-4b10-a294-1215058766e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.131226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb082f-5d69-4b10-a294-1215058766e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.131509 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb082f-5d69-4b10-a294-1215058766e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.131777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb082f-5d69-4b10-a294-1215058766e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.132438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb082f-5d69-4b10-a294-1215058766e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:35.132778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb082f-5d69-4b10-a294-1215058766e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.134412 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb082f-5d69-4b10-a294-1215058766e1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.135402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a33aab-d26f-4148-80d7-2421d2bf8264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.136011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a33aab-d26f-4148-80d7-2421d2bf8264 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.136258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a33aab-d26f-4148-80d7-2421d2bf8264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.136524 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a33aab-d26f-4148-80d7-2421d2bf8264 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.137188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a33aab-d26f-4148-80d7-2421d2bf8264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:35.137491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a33aab-d26f-4148-80d7-2421d2bf8264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.139073 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a33aab-d26f-4148-80d7-2421d2bf8264 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.147527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 774/1565] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:35.150822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff886ea-cc58-4a83-914a-9c8d5356a701 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.151111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff886ea-cc58-4a83-914a-9c8d5356a701 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.151390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff886ea-cc58-4a83-914a-9c8d5356a701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.152484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 792/1566] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:35.153382 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a33aab-d26f-4148-80d7-2421d2bf8264 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.164403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a33aab-d26f-4148-80d7-2421d2bf8264 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.169045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1788f43-7794-42fe-82ca-b9de4eefba5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.169669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1788f43-7794-42fe-82ca-b9de4eefba5a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.169948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1788f43-7794-42fe-82ca-b9de4eefba5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.170218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1788f43-7794-42fe-82ca-b9de4eefba5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.170895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1788f43-7794-42fe-82ca-b9de4eefba5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:35.171231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1788f43-7794-42fe-82ca-b9de4eefba5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.172871 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1788f43-7794-42fe-82ca-b9de4eefba5a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.176942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9ff886ea-cc58-4a83-914a-9c8d5356a701 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:35.178180 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9ff886ea-cc58-4a83-914a-9c8d5356a701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:35.186042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 793/1567] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:35.186227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 775/1568] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:35.190317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1788f43-7794-42fe-82ca-b9de4eefba5a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.190511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff886ea-cc58-4a83-914a-9c8d5356a701 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.198679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff886ea-cc58-4a83-914a-9c8d5356a701 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.201787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1788f43-7794-42fe-82ca-b9de4eefba5a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.203490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fda084-95ec-4342-9458-742f5dd61410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.204185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda084-95ec-4342-9458-742f5dd61410 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.204528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda084-95ec-4342-9458-742f5dd61410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.204889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda084-95ec-4342-9458-742f5dd61410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.205656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fda084-95ec-4342-9458-742f5dd61410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.206474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c9f33-979a-44be-8c45-b09bd332ba44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.207258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c9f33-979a-44be-8c45-b09bd332ba44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.207534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c9f33-979a-44be-8c45-b09bd332ba44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.207802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c9f33-979a-44be-8c45-b09bd332ba44 admin admin] PATH_INFO: `/v3/projects/96fea11cfdd748bab1cdea46b073dc89/users/85d8138d44bc40dd8f63ee4916ffbce9/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.207915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fda084-95ec-4342-9458-742f5dd61410 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.218205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 776/1569] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:35.218836 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fda084-95ec-4342-9458-742f5dd61410 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.222694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c9f33-979a-44be-8c45-b09bd332ba44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96fea11cfdd748bab1cdea46b073dc89, user_id=85d8138d44bc40dd8f63ee4916ffbce9, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.223730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: WARNING py.warnings [None req-5a0c9f33-979a-44be-8c45-b09bd332ba44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:32:35.223730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: warnings.warn(msg) Aug 24 12:32:35.223730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: Aug 24 12:32:35.225707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c9f33-979a-44be-8c45-b09bd332ba44 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.231596 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fda084-95ec-4342-9458-742f5dd61410 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.236431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f3e4b-a1a8-4ddd-80c3-265fba724f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.237061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f3e4b-a1a8-4ddd-80c3-265fba724f50 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.237353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f3e4b-a1a8-4ddd-80c3-265fba724f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.238295 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f3e4b-a1a8-4ddd-80c3-265fba724f50 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.238916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f3e4b-a1a8-4ddd-80c3-265fba724f50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.243764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 794/1570] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:35 2022] PUT /identity/v3/projects/96fea11cfdd748bab1cdea46b073dc89/users/85d8138d44bc40dd8f63ee4916ffbce9/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:35.243764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c9f33-979a-44be-8c45-b09bd332ba44 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.244378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f3e4b-a1a8-4ddd-80c3-265fba724f50 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.267068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 777/1571] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/users => generated 284 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:35.268153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f3e4b-a1a8-4ddd-80c3-265fba724f50 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.278347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f3e4b-a1a8-4ddd-80c3-265fba724f50 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.304433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c9f33-979a-44be-8c45-b09bd332ba44 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.309423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6e868-b946-4946-90a3-af54487ea597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.310080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6e868-b946-4946-90a3-af54487ea597 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.310382 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6e868-b946-4946-90a3-af54487ea597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.310652 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6e868-b946-4946-90a3-af54487ea597 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.311291 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6e868-b946-4946-90a3-af54487ea597 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:35.311697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6e868-b946-4946-90a3-af54487ea597 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.313234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6e868-b946-4946-90a3-af54487ea597 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.325281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 795/1572] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:35.326134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6e868-b946-4946-90a3-af54487ea597 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.338875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717979e-40c9-4340-9211-f97679fb1649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.339504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717979e-40c9-4340-9211-f97679fb1649 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.339781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717979e-40c9-4340-9211-f97679fb1649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.340054 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717979e-40c9-4340-9211-f97679fb1649 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.340712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717979e-40c9-4340-9211-f97679fb1649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.342374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6e868-b946-4946-90a3-af54487ea597 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.342688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717979e-40c9-4340-9211-f97679fb1649 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.347003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8d690-3c65-4c3a-ab79-945cb6bc13da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.347647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8d690-3c65-4c3a-ab79-945cb6bc13da admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.347923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8d690-3c65-4c3a-ab79-945cb6bc13da admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.348212 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8d690-3c65-4c3a-ab79-945cb6bc13da admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.348938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8d690-3c65-4c3a-ab79-945cb6bc13da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.350823 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8d690-3c65-4c3a-ab79-945cb6bc13da admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.372891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:35.376554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 778/1573] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:35.377465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717979e-40c9-4340-9211-f97679fb1649 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.383694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 796/1574] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:35.383694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8d690-3c65-4c3a-ab79-945cb6bc13da admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.426383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:35.448549 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8d690-3c65-4c3a-ab79-945cb6bc13da admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.453148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab67d1-7401-4648-b76d-0cff4e27b4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.453762 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab67d1-7401-4648-b76d-0cff4e27b4e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.454037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab67d1-7401-4648-b76d-0cff4e27b4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.454303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab67d1-7401-4648-b76d-0cff4e27b4e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.454930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab67d1-7401-4648-b76d-0cff4e27b4e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.457086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab67d1-7401-4648-b76d-0cff4e27b4e6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.457509 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717979e-40c9-4340-9211-f97679fb1649 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.462308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99eda4-7e1b-46f4-aaff-6b7aeca741c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.462939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99eda4-7e1b-46f4-aaff-6b7aeca741c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.463211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99eda4-7e1b-46f4-aaff-6b7aeca741c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.463477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99eda4-7e1b-46f4-aaff-6b7aeca741c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.464122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99eda4-7e1b-46f4-aaff-6b7aeca741c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:35.464448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99eda4-7e1b-46f4-aaff-6b7aeca741c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.466091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99eda4-7e1b-46f4-aaff-6b7aeca741c5 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.478861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 779/1575] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:35.479299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99eda4-7e1b-46f4-aaff-6b7aeca741c5 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.486113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 797/1576] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:35.487960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dcb8e-0c8e-40e2-aaa5-da0acda1f67e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.488409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dcb8e-0c8e-40e2-aaa5-da0acda1f67e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.488859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dcb8e-0c8e-40e2-aaa5-da0acda1f67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.489837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99eda4-7e1b-46f4-aaff-6b7aeca741c5 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.508458 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-439dcb8e-0c8e-40e2-aaa5-da0acda1f67e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:35.510117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-439dcb8e-0c8e-40e2-aaa5-da0acda1f67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:35.545880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada15d6-ed65-4e4d-8010-c77f740f02ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.546547 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada15d6-ed65-4e4d-8010-c77f740f02ef admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.546824 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada15d6-ed65-4e4d-8010-c77f740f02ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.547092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada15d6-ed65-4e4d-8010-c77f740f02ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.547763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada15d6-ed65-4e4d-8010-c77f740f02ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.549431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada15d6-ed65-4e4d-8010-c77f740f02ef admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.559760 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 780/1577] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:35.560816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada15d6-ed65-4e4d-8010-c77f740f02ef admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.571720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada15d6-ed65-4e4d-8010-c77f740f02ef admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.572726 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 798/1578] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:35.573995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439dcb8e-0c8e-40e2-aaa5-da0acda1f67e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.576958 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc12b72-ee56-4b2d-a6a0-be4448995680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.577209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc12b72-ee56-4b2d-a6a0-be4448995680 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.577432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc12b72-ee56-4b2d-a6a0-be4448995680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.577844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc12b72-ee56-4b2d-a6a0-be4448995680 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.578579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc12b72-ee56-4b2d-a6a0-be4448995680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.580106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc12b72-ee56-4b2d-a6a0-be4448995680 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.586400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439dcb8e-0c8e-40e2-aaa5-da0acda1f67e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.591164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7975384-b220-49b0-bd44-b520298b6c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.591537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 781/1579] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:35.591808 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7975384-b220-49b0-bd44-b520298b6c63 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.592085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7975384-b220-49b0-bd44-b520298b6c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.592326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc12b72-ee56-4b2d-a6a0-be4448995680 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.592432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7975384-b220-49b0-bd44-b520298b6c63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.593108 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7975384-b220-49b0-bd44-b520298b6c63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.595175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7975384-b220-49b0-bd44-b520298b6c63 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.602098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc12b72-ee56-4b2d-a6a0-be4448995680 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.606699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb272175-ab79-499e-9c56-d235ef1deac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.607267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb272175-ab79-499e-9c56-d235ef1deac7 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.607543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb272175-ab79-499e-9c56-d235ef1deac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.607811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb272175-ab79-499e-9c56-d235ef1deac7 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.608423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb272175-ab79-499e-9c56-d235ef1deac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.610360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb272175-ab79-499e-9c56-d235ef1deac7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.627254 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 799/1580] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/projects => generated 399 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:35.628817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7975384-b220-49b0-bd44-b520298b6c63 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.637914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 782/1581] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:35.637914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb272175-ab79-499e-9c56-d235ef1deac7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.646393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb272175-ab79-499e-9c56-d235ef1deac7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.697710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7975384-b220-49b0-bd44-b520298b6c63 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.703001 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f1be0-d1a5-41ff-818d-d444738516cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.703618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f1be0-d1a5-41ff-818d-d444738516cc admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.703895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f1be0-d1a5-41ff-818d-d444738516cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.704165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f1be0-d1a5-41ff-818d-d444738516cc admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.704263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a532e8e-5a50-4d32-9571-097ea2cde53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.704920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a532e8e-5a50-4d32-9571-097ea2cde53e admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.705075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f1be0-d1a5-41ff-818d-d444738516cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.705522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a532e8e-5a50-4d32-9571-097ea2cde53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.705522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a532e8e-5a50-4d32-9571-097ea2cde53e admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.706150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a532e8e-5a50-4d32-9571-097ea2cde53e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.707269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f1be0-d1a5-41ff-818d-d444738516cc admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.708198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a532e8e-5a50-4d32-9571-097ea2cde53e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.718008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:35.735176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 800/1582] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:35.735176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f1be0-d1a5-41ff-818d-d444738516cc admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.736046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 783/1583] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:35.736792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a532e8e-5a50-4d32-9571-097ea2cde53e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.744417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f1be0-d1a5-41ff-818d-d444738516cc admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.757327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:35.800519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cea2e-c395-4b67-88c7-f4ec2183d759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.800802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a532e8e-5a50-4d32-9571-097ea2cde53e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.801111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cea2e-c395-4b67-88c7-f4ec2183d759 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.801491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cea2e-c395-4b67-88c7-f4ec2183d759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.801769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cea2e-c395-4b67-88c7-f4ec2183d759 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.802440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cea2e-c395-4b67-88c7-f4ec2183d759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.804403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cea2e-c395-4b67-88c7-f4ec2183d759 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.805770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62947d81-9b7a-4929-a22c-085e714b4975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.806433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62947d81-9b7a-4929-a22c-085e714b4975 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.806772 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62947d81-9b7a-4929-a22c-085e714b4975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.807094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62947d81-9b7a-4929-a22c-085e714b4975 admin admin] PATH_INFO: `/v3/projects/27c1d9bc37534d32b9bc0212a80e672b/users/63abae805aae45509764b9747f248ad2/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.823776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62947d81-9b7a-4929-a22c-085e714b4975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c1d9bc37534d32b9bc0212a80e672b, user_id=63abae805aae45509764b9747f248ad2, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.826508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62947d81-9b7a-4929-a22c-085e714b4975 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.828680 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 801/1584] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:35.829436 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cea2e-c395-4b67-88c7-f4ec2183d759 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.831031 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f96cea2e-c395-4b67-88c7-f4ec2183d759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:35.845670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 784/1585] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:35 2022] PUT /identity/v3/projects/27c1d9bc37534d32b9bc0212a80e672b/users/63abae805aae45509764b9747f248ad2/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:35.846495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62947d81-9b7a-4929-a22c-085e714b4975 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.891566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cea2e-c395-4b67-88c7-f4ec2183d759 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.907348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62947d81-9b7a-4929-a22c-085e714b4975 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.912147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244c69a-2da2-4a82-be2b-ad403c86f2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.912809 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244c69a-2da2-4a82-be2b-ad403c86f2db admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.913085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244c69a-2da2-4a82-be2b-ad403c86f2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.913356 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244c69a-2da2-4a82-be2b-ad403c86f2db admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.914002 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244c69a-2da2-4a82-be2b-ad403c86f2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.915627 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244c69a-2da2-4a82-be2b-ad403c86f2db admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.925937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 785/1586] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:35.926913 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244c69a-2da2-4a82-be2b-ad403c86f2db admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.937052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244c69a-2da2-4a82-be2b-ad403c86f2db admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.941313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673ce1b-44f3-4b5f-9d45-88f84534c4bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.942311 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673ce1b-44f3-4b5f-9d45-88f84534c4bb service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.942590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673ce1b-44f3-4b5f-9d45-88f84534c4bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.942859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673ce1b-44f3-4b5f-9d45-88f84534c4bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.943109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ac78a-04bd-463a-906e-ca5532947eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.943704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ac78a-04bd-463a-906e-ca5532947eba admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.943991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ac78a-04bd-463a-906e-ca5532947eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.944261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ac78a-04bd-463a-906e-ca5532947eba admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.944920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ac78a-04bd-463a-906e-ca5532947eba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.947030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ac78a-04bd-463a-906e-ca5532947eba admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.951960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673ce1b-44f3-4b5f-9d45-88f84534c4bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.953430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673ce1b-44f3-4b5f-9d45-88f84534c4bb service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.964841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 802/1587] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:35.965757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673ce1b-44f3-4b5f-9d45-88f84534c4bb service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.974212 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 786/1588] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/users => generated 300 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:35.974212 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ac78a-04bd-463a-906e-ca5532947eba admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.975380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673ce1b-44f3-4b5f-9d45-88f84534c4bb service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.982475 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d0b13-41bc-4182-8dc3-1a082ef82c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.983175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0b13-41bc-4182-8dc3-1a082ef82c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.983561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0b13-41bc-4182-8dc3-1a082ef82c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.983888 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0b13-41bc-4182-8dc3-1a082ef82c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.985039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d0b13-41bc-4182-8dc3-1a082ef82c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.986360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d0b13-41bc-4182-8dc3-1a082ef82c96 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.987723 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ac78a-04bd-463a-906e-ca5532947eba admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.996452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99452ee9-fba2-449b-9100-a13834fcb7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.997166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99452ee9-fba2-449b-9100-a13834fcb7ae admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.997476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99452ee9-fba2-449b-9100-a13834fcb7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.997752 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99452ee9-fba2-449b-9100-a13834fcb7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.998184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 803/1589] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:35.998431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99452ee9-fba2-449b-9100-a13834fcb7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.000194 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99452ee9-fba2-449b-9100-a13834fcb7ae admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.001661 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d0b13-41bc-4182-8dc3-1a082ef82c96 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.013278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d0b13-41bc-4182-8dc3-1a082ef82c96 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.013693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 787/1590] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:36.014535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99452ee9-fba2-449b-9100-a13834fcb7ae admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.022336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa731384-555e-4040-b2c2-2a0325fe2f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.023166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa731384-555e-4040-b2c2-2a0325fe2f33 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.023628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa731384-555e-4040-b2c2-2a0325fe2f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.024103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa731384-555e-4040-b2c2-2a0325fe2f33 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.025029 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99452ee9-fba2-449b-9100-a13834fcb7ae admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.025306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa731384-555e-4040-b2c2-2a0325fe2f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.028104 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa731384-555e-4040-b2c2-2a0325fe2f33 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.029987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab30e8-7d1d-4707-9db1-8e0fd5f7b127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.030711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab30e8-7d1d-4707-9db1-8e0fd5f7b127 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.031170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab30e8-7d1d-4707-9db1-8e0fd5f7b127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.032021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab30e8-7d1d-4707-9db1-8e0fd5f7b127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.032261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab30e8-7d1d-4707-9db1-8e0fd5f7b127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.034345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab30e8-7d1d-4707-9db1-8e0fd5f7b127 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.046331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 788/1591] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:36.047847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0ccbb-3706-4b6d-bee4-d983cbabb9ad None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.048226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0ccbb-3706-4b6d-bee4-d983cbabb9ad None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.048596 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0ccbb-3706-4b6d-bee4-d983cbabb9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.054769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 804/1592] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/users => generated 303 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:36.055683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa731384-555e-4040-b2c2-2a0325fe2f33 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.068206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa731384-555e-4040-b2c2-2a0325fe2f33 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.073924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f159657-5605-43e8-afb8-ab5abf5363d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.074582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f159657-5605-43e8-afb8-ab5abf5363d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.074865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f159657-5605-43e8-afb8-ab5abf5363d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.075136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f159657-5605-43e8-afb8-ab5abf5363d7 admin admin] PATH_INFO: `/v3/projects/cfeddd326a4045a0a724223bcb0c9487/users/d6eddc93f36e4af5ba0a85a420e7a6e9/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.081312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6ae0ccbb-3706-4b6d-bee4-d983cbabb9ad None None] MFA Rules not processed for user `63abae805aae45509764b9747f248ad2`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:36.082771 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6ae0ccbb-3706-4b6d-bee4-d983cbabb9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:36.089426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:36.095412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f159657-5605-43e8-afb8-ab5abf5363d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfeddd326a4045a0a724223bcb0c9487, user_id=d6eddc93f36e4af5ba0a85a420e7a6e9, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.100700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f159657-5605-43e8-afb8-ab5abf5363d7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.120005 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 805/1593] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:36 2022] PUT /identity/v3/projects/cfeddd326a4045a0a724223bcb0c9487/users/d6eddc93f36e4af5ba0a85a420e7a6e9/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:36.120747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f159657-5605-43e8-afb8-ab5abf5363d7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.140706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:36.177409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 789/1594] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:36.177409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0ccbb-3706-4b6d-bee4-d983cbabb9ad None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.186408 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0ccbb-3706-4b6d-bee4-d983cbabb9ad None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.241325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f159657-5605-43e8-afb8-ab5abf5363d7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.247457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a80dc0-3626-4cea-8ba1-57e176bc24d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.248372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a80dc0-3626-4cea-8ba1-57e176bc24d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.248672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a80dc0-3626-4cea-8ba1-57e176bc24d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.248957 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a80dc0-3626-4cea-8ba1-57e176bc24d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.249764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a80dc0-3626-4cea-8ba1-57e176bc24d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.251776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a80dc0-3626-4cea-8ba1-57e176bc24d3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.262202 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99ee27-bc23-4c91-a716-f78f89424f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.262817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99ee27-bc23-4c91-a716-f78f89424f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.263093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99ee27-bc23-4c91-a716-f78f89424f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.263740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99ee27-bc23-4c91-a716-f78f89424f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.264004 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99ee27-bc23-4c91-a716-f78f89424f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.265795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99ee27-bc23-4c91-a716-f78f89424f8b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.270258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 806/1595] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:36.270258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a80dc0-3626-4cea-8ba1-57e176bc24d3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.279863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 790/1596] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:36.279863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99ee27-bc23-4c91-a716-f78f89424f8b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.282333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a80dc0-3626-4cea-8ba1-57e176bc24d3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.288337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54777a-1a81-42d9-b756-367a9cda3e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.288337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54777a-1a81-42d9-b756-367a9cda3e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.288887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54777a-1a81-42d9-b756-367a9cda3e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.289258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54777a-1a81-42d9-b756-367a9cda3e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.289820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99ee27-bc23-4c91-a716-f78f89424f8b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.290101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54777a-1a81-42d9-b756-367a9cda3e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.292246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54777a-1a81-42d9-b756-367a9cda3e7b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.297469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599019d-eb32-4669-8501-922e10d9c281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.298227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599019d-eb32-4669-8501-922e10d9c281 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.299359 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599019d-eb32-4669-8501-922e10d9c281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.299466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599019d-eb32-4669-8501-922e10d9c281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.300227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599019d-eb32-4669-8501-922e10d9c281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.303202 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599019d-eb32-4669-8501-922e10d9c281 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.306163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 807/1597] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:36.307410 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54777a-1a81-42d9-b756-367a9cda3e7b admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.319841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54777a-1a81-42d9-b756-367a9cda3e7b admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.327428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 791/1598] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:36.327428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe5261-22e8-4a11-8cbd-8c730270592c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.327428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe5261-22e8-4a11-8cbd-8c730270592c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.327428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe5261-22e8-4a11-8cbd-8c730270592c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.332727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06211a3d-14fc-4267-b91a-bd36749c5b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.333970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06211a3d-14fc-4267-b91a-bd36749c5b85 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.334703 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06211a3d-14fc-4267-b91a-bd36749c5b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.335483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06211a3d-14fc-4267-b91a-bd36749c5b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.337588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06211a3d-14fc-4267-b91a-bd36749c5b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.338938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06211a3d-14fc-4267-b91a-bd36749c5b85 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.351830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 808/1599] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:36.355313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84217258-698a-4388-89b2-16774b19c3be None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.355561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84217258-698a-4388-89b2-16774b19c3be None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.355832 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84217258-698a-4388-89b2-16774b19c3be None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.357169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 809/1600] 10.222.0.27 () {54 vars in 962 bytes} [Wed Aug 24 12:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:36.362628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84217258-698a-4388-89b2-16774b19c3be None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.365934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8dbe5261-22e8-4a11-8cbd-8c730270592c None None] MFA Rules not processed for user `d6eddc93f36e4af5ba0a85a420e7a6e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:36.367810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8dbe5261-22e8-4a11-8cbd-8c730270592c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:36.375821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84217258-698a-4388-89b2-16774b19c3be None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.381713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4618f-2b38-46b1-8304-5dca8ffc4b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.382513 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4618f-2b38-46b1-8304-5dca8ffc4b71 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.383179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4618f-2b38-46b1-8304-5dca8ffc4b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.383179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4618f-2b38-46b1-8304-5dca8ffc4b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.383740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4618f-2b38-46b1-8304-5dca8ffc4b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.385709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4618f-2b38-46b1-8304-5dca8ffc4b71 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.400354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 810/1601] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:36.401379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4618f-2b38-46b1-8304-5dca8ffc4b71 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.415476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4618f-2b38-46b1-8304-5dca8ffc4b71 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.425482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d366d28-c110-4844-a8fc-4bb339f2e3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.426341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d366d28-c110-4844-a8fc-4bb339f2e3fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.426626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d366d28-c110-4844-a8fc-4bb339f2e3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.426899 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d366d28-c110-4844-a8fc-4bb339f2e3fd admin admin] PATH_INFO: `/v3/projects/a646ac1b653b49fb80c05ad567f29710/users/8e1c70920a8a43999c308c8f595b4528/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.448442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d366d28-c110-4844-a8fc-4bb339f2e3fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a646ac1b653b49fb80c05ad567f29710, user_id=8e1c70920a8a43999c308c8f595b4528, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.450919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 792/1602] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:36.451985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe5261-22e8-4a11-8cbd-8c730270592c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.466176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe5261-22e8-4a11-8cbd-8c730270592c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.466748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d366d28-c110-4844-a8fc-4bb339f2e3fd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.471360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0d989-6415-40b0-9140-fa6c4300a425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.472590 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0d989-6415-40b0-9140-fa6c4300a425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.472590 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0d989-6415-40b0-9140-fa6c4300a425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.472916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0d989-6415-40b0-9140-fa6c4300a425 admin admin] PATH_INFO: `/v3/projects/3276a7bc6bb84ac7953d2d46c4298a23/users/e34cbcf69b4b4828a3bca077fc8d0694/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.485602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 811/1603] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:36 2022] PUT /identity/v3/projects/a646ac1b653b49fb80c05ad567f29710/users/8e1c70920a8a43999c308c8f595b4528/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:36.486272 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d366d28-c110-4844-a8fc-4bb339f2e3fd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.493103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0d989-6415-40b0-9140-fa6c4300a425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3276a7bc6bb84ac7953d2d46c4298a23, user_id=e34cbcf69b4b4828a3bca077fc8d0694, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.495911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0d989-6415-40b0-9140-fa6c4300a425 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.513501 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 793/1604] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:36 2022] PUT /identity/v3/projects/3276a7bc6bb84ac7953d2d46c4298a23/users/e34cbcf69b4b4828a3bca077fc8d0694/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:36.514434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0d989-6415-40b0-9140-fa6c4300a425 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.562358 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d366d28-c110-4844-a8fc-4bb339f2e3fd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.569530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:36.596406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0d989-6415-40b0-9140-fa6c4300a425 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.600859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:36.603348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdba61c-6802-43e3-adbe-6ede98e0ea40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.604018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdba61c-6802-43e3-adbe-6ede98e0ea40 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.604536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdba61c-6802-43e3-adbe-6ede98e0ea40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.604938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdba61c-6802-43e3-adbe-6ede98e0ea40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.605754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdba61c-6802-43e3-adbe-6ede98e0ea40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.607403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdba61c-6802-43e3-adbe-6ede98e0ea40 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.620176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 794/1605] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:36.624761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c8c0e-8fda-4b1f-bd9f-43634137d3c6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.625048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c8c0e-8fda-4b1f-bd9f-43634137d3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.625480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c8c0e-8fda-4b1f-bd9f-43634137d3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.635841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698ab7d-99c3-44d3-9b78-93f26c78537d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.636809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698ab7d-99c3-44d3-9b78-93f26c78537d admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.637250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698ab7d-99c3-44d3-9b78-93f26c78537d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.637560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698ab7d-99c3-44d3-9b78-93f26c78537d admin admin] PATH_INFO: `/v3/projects/75590a43f50b48699fc0d5a892a99544/users/b4a617fc13534a89ad50a5fdc7bf769a/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.662424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698ab7d-99c3-44d3-9b78-93f26c78537d admin admin] RBAC: Authorizing `identity:create_grant(project_id=75590a43f50b48699fc0d5a892a99544, user_id=b4a617fc13534a89ad50a5fdc7bf769a, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.665371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698ab7d-99c3-44d3-9b78-93f26c78537d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.682852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 812/1606] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:36 2022] PUT /identity/v3/projects/75590a43f50b48699fc0d5a892a99544/users/b4a617fc13534a89ad50a5fdc7bf769a/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:36.684990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698ab7d-99c3-44d3-9b78-93f26c78537d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.687541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-433c8c0e-8fda-4b1f-bd9f-43634137d3c6 None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:36.688882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-433c8c0e-8fda-4b1f-bd9f-43634137d3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:36.773002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698ab7d-99c3-44d3-9b78-93f26c78537d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.779226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23ff6d-3ad8-45de-85ee-6f34a7cf736d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.780150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23ff6d-3ad8-45de-85ee-6f34a7cf736d admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.780283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23ff6d-3ad8-45de-85ee-6f34a7cf736d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.780579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23ff6d-3ad8-45de-85ee-6f34a7cf736d admin admin] PATH_INFO: `/v3/projects/106a7bed7da74646ab46e21b34d47f10/users/b71e3ef17a354f0da5357b119b053bcc/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.808326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 795/1607] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Aug 24 12:32:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:36.810939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23ff6d-3ad8-45de-85ee-6f34a7cf736d admin admin] RBAC: Authorizing `identity:create_grant(project_id=106a7bed7da74646ab46e21b34d47f10, user_id=b71e3ef17a354f0da5357b119b053bcc, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.814204 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23ff6d-3ad8-45de-85ee-6f34a7cf736d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.818490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c8c0e-8fda-4b1f-bd9f-43634137d3c6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:36.844240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 813/1608] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:36 2022] PUT /identity/v3/projects/106a7bed7da74646ab46e21b34d47f10/users/b71e3ef17a354f0da5357b119b053bcc/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:36.845948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceaa20-366a-43dd-b80c-769698b8e5d2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.846455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceaa20-366a-43dd-b80c-769698b8e5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.847858 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceaa20-366a-43dd-b80c-769698b8e5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.875816 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-efceaa20-366a-43dd-b80c-769698b8e5d2 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:36.877727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-efceaa20-366a-43dd-b80c-769698b8e5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:36.892710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c8c0e-8fda-4b1f-bd9f-43634137d3c6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.965044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 814/1609] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:36.970080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff8d76-908c-4dd3-94cf-244607a084c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.970788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff8d76-908c-4dd3-94cf-244607a084c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.971104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff8d76-908c-4dd3-94cf-244607a084c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.971415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff8d76-908c-4dd3-94cf-244607a084c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.976772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a4dd4-b037-4da3-a387-4d7ce80b0c6f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.976772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a4dd4-b037-4da3-a387-4d7ce80b0c6f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.976772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a4dd4-b037-4da3-a387-4d7ce80b0c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.003747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b60a4dd4-b037-4da3-a387-4d7ce80b0c6f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:37.005990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b60a4dd4-b037-4da3-a387-4d7ce80b0c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.010169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:37.022155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 815/1610] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:37.023275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a4dd4-b037-4da3-a387-4d7ce80b0c6f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.051243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a4dd4-b037-4da3-a387-4d7ce80b0c6f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.057299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff8d76-908c-4dd3-94cf-244607a084c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.058085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b516706-ce2a-4f2d-84ea-c4021ebc23e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.058838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b516706-ce2a-4f2d-84ea-c4021ebc23e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.059185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b516706-ce2a-4f2d-84ea-c4021ebc23e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.059303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff8d76-908c-4dd3-94cf-244607a084c5 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.059589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b516706-ce2a-4f2d-84ea-c4021ebc23e5 admin admin] PATH_INFO: `/v3/projects/83e6b8558a3146279d84792f251a3dd2/users/68e836eb8ee7475788d85d2f95329b20/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.071082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 796/1611] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:37.072552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746c458-39a1-49ab-b492-c656d248ac2f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.072969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746c458-39a1-49ab-b492-c656d248ac2f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.073312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746c458-39a1-49ab-b492-c656d248ac2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.098850 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b516706-ce2a-4f2d-84ea-c4021ebc23e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83e6b8558a3146279d84792f251a3dd2, user_id=68e836eb8ee7475788d85d2f95329b20, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.101662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f746c458-39a1-49ab-b492-c656d248ac2f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:37.103108 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b516706-ce2a-4f2d-84ea-c4021ebc23e5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.104112 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f746c458-39a1-49ab-b492-c656d248ac2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.122643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 816/1612] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:37 2022] PUT /identity/v3/projects/83e6b8558a3146279d84792f251a3dd2/users/68e836eb8ee7475788d85d2f95329b20/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:37.123891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b516706-ce2a-4f2d-84ea-c4021ebc23e5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.131602 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 797/1613] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:37.134145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c780f2-ef05-447e-887f-27d1848d8380 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.135245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c780f2-ef05-447e-887f-27d1848d8380 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.135585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c780f2-ef05-447e-887f-27d1848d8380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.164935 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:37.190603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b4c780f2-ef05-447e-887f-27d1848d8380 None None] MFA Rules not processed for user `b71e3ef17a354f0da5357b119b053bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:37.195434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b4c780f2-ef05-447e-887f-27d1848d8380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.256604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b516706-ce2a-4f2d-84ea-c4021ebc23e5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.264639 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9c9c6-2405-45ba-af90-1936c581dc56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.265386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9c9c6-2405-45ba-af90-1936c581dc56 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.265729 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9c9c6-2405-45ba-af90-1936c581dc56 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.266067 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9c9c6-2405-45ba-af90-1936c581dc56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.336513 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9c9c6-2405-45ba-af90-1936c581dc56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.338279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9c9c6-2405-45ba-af90-1936c581dc56 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.360064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 798/1614] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:37.360573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 817/1615] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:37.361770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9c9c6-2405-45ba-af90-1936c581dc56 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.361946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c780f2-ef05-447e-887f-27d1848d8380 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.448675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9c9c6-2405-45ba-af90-1936c581dc56 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.453099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12200a6-8320-4d0e-9e50-da50a9418e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.453707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12200a6-8320-4d0e-9e50-da50a9418e09 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.453985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12200a6-8320-4d0e-9e50-da50a9418e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.454262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12200a6-8320-4d0e-9e50-da50a9418e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.517779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c780f2-ef05-447e-887f-27d1848d8380 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.521623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12200a6-8320-4d0e-9e50-da50a9418e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.522185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d6c98-0e65-4970-8194-f9a0f8946c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.522800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d6c98-0e65-4970-8194-f9a0f8946c40 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.523071 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d6c98-0e65-4970-8194-f9a0f8946c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.523253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12200a6-8320-4d0e-9e50-da50a9418e09 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.523430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d6c98-0e65-4970-8194-f9a0f8946c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.527274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:37.545119 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d6c98-0e65-4970-8194-f9a0f8946c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.545731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 818/1616] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:37.547090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d6c98-0e65-4970-8194-f9a0f8946c40 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.561638 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab568cc4-36fa-4aec-8ae7-ff2d7d363100 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.562112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab568cc4-36fa-4aec-8ae7-ff2d7d363100 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.562551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab568cc4-36fa-4aec-8ae7-ff2d7d363100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.572519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 799/1617] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:37.574599 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d6c98-0e65-4970-8194-f9a0f8946c40 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.594572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d6c98-0e65-4970-8194-f9a0f8946c40 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.602480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf7bd5-e189-44f5-86a8-aefc1ebb6f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.603774 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf7bd5-e189-44f5-86a8-aefc1ebb6f67 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.604521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf7bd5-e189-44f5-86a8-aefc1ebb6f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.606555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf7bd5-e189-44f5-86a8-aefc1ebb6f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.606752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ab568cc4-36fa-4aec-8ae7-ff2d7d363100 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:37.607999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ab568cc4-36fa-4aec-8ae7-ff2d7d363100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.627794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 819/1618] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:37.630790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab568cc4-36fa-4aec-8ae7-ff2d7d363100 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.632724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf7bd5-e189-44f5-86a8-aefc1ebb6f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.634364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf7bd5-e189-44f5-86a8-aefc1ebb6f67 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.652161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab568cc4-36fa-4aec-8ae7-ff2d7d363100 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.655018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 800/1619] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:37.667005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf7bd5-e189-44f5-86a8-aefc1ebb6f67 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.669147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0526ac-202b-4613-b9b1-6ebb6e30dee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.670102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0526ac-202b-4613-b9b1-6ebb6e30dee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.670559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0526ac-202b-4613-b9b1-6ebb6e30dee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.671111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0526ac-202b-4613-b9b1-6ebb6e30dee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.671937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0526ac-202b-4613-b9b1-6ebb6e30dee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.674108 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0526ac-202b-4613-b9b1-6ebb6e30dee5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.695720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf7bd5-e189-44f5-86a8-aefc1ebb6f67 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.707678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bbac1-9dfc-4ceb-a565-6cfd5e1b457d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.709804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bbac1-9dfc-4ceb-a565-6cfd5e1b457d service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.711128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bbac1-9dfc-4ceb-a565-6cfd5e1b457d service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.711403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bbac1-9dfc-4ceb-a565-6cfd5e1b457d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.735983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bbac1-9dfc-4ceb-a565-6cfd5e1b457d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.737684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bbac1-9dfc-4ceb-a565-6cfd5e1b457d service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.743935 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 820/1620] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/projects => generated 401 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:37.748289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0526ac-202b-4613-b9b1-6ebb6e30dee5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.766493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 801/1621] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:37.888478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0526ac-202b-4613-b9b1-6ebb6e30dee5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.900218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b8315-fc4c-4b80-9868-c23c6cb3c0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.901203 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b8315-fc4c-4b80-9868-c23c6cb3c0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.901631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b8315-fc4c-4b80-9868-c23c6cb3c0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.902069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b8315-fc4c-4b80-9868-c23c6cb3c0b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.905145 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b8315-fc4c-4b80-9868-c23c6cb3c0b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.905145 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b8315-fc4c-4b80-9868-c23c6cb3c0b0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.958055 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 821/1622] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/users => generated 307 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:37.961957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bbac1-9dfc-4ceb-a565-6cfd5e1b457d service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.982045 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bbac1-9dfc-4ceb-a565-6cfd5e1b457d service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.989100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa2d4b-a4ff-426e-8f37-08de7029f60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.989803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa2d4b-a4ff-426e-8f37-08de7029f60e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.990148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa2d4b-a4ff-426e-8f37-08de7029f60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.990417 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa2d4b-a4ff-426e-8f37-08de7029f60e admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.991235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa2d4b-a4ff-426e-8f37-08de7029f60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.994514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa2d4b-a4ff-426e-8f37-08de7029f60e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:38.017829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 802/1623] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:38.022328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b8315-fc4c-4b80-9868-c23c6cb3c0b0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:38.038151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b8315-fc4c-4b80-9868-c23c6cb3c0b0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:38.044642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335981f-a9e8-4017-9a94-37c27459db75 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:38.044985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335981f-a9e8-4017-9a94-37c27459db75 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:38.045442 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335981f-a9e8-4017-9a94-37c27459db75 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:38.054749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 803/1624] 10.222.0.27 () {54 vars in 912 bytes} [Wed Aug 24 12:32:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:38.064982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a2135-b4cc-4ef1-9e79-48aa3c2f085a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:38.072186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a2135-b4cc-4ef1-9e79-48aa3c2f085a admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:38.072423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac07032-3bb1-4a98-8e48-fb8b38800849 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:38.072601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac07032-3bb1-4a98-8e48-fb8b38800849 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:38.073009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac07032-3bb1-4a98-8e48-fb8b38800849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:38.073832 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a2135-b4cc-4ef1-9e79-48aa3c2f085a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:38.075213 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a2135-b4cc-4ef1-9e79-48aa3c2f085a admin admin] PATH_INFO: `/v3/projects/95b570ff2f5e4de3b1567dbb54582086/users/e0e00f8bc62940acab31f0d6d1c82f43/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:38.107832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-eac07032-3bb1-4a98-8e48-fb8b38800849 None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:38.109595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eac07032-3bb1-4a98-8e48-fb8b38800849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:38.118052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:38.140038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a2135-b4cc-4ef1-9e79-48aa3c2f085a admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b570ff2f5e4de3b1567dbb54582086, user_id=e0e00f8bc62940acab31f0d6d1c82f43, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:38.147176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a2135-b4cc-4ef1-9e79-48aa3c2f085a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:38.168993 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 822/1625] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:38 2022] PUT /identity/v3/projects/95b570ff2f5e4de3b1567dbb54582086/users/e0e00f8bc62940acab31f0d6d1c82f43/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:38.173408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a2135-b4cc-4ef1-9e79-48aa3c2f085a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:38.368540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 804/1626] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:32:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:38.390334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac07032-3bb1-4a98-8e48-fb8b38800849 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:38.516474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a2135-b4cc-4ef1-9e79-48aa3c2f085a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:38.518755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac07032-3bb1-4a98-8e48-fb8b38800849 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:38.545180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98551293-5bf0-459e-8498-f821201d7cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:38.546935 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98551293-5bf0-459e-8498-f821201d7cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:38.548493 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98551293-5bf0-459e-8498-f821201d7cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:38.549904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98551293-5bf0-459e-8498-f821201d7cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:38.553268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98551293-5bf0-459e-8498-f821201d7cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:38.562258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98551293-5bf0-459e-8498-f821201d7cb2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:38.592775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 823/1627] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:38.601050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98551293-5bf0-459e-8498-f821201d7cb2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:38.623465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98551293-5bf0-459e-8498-f821201d7cb2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:38.633687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc1ebb-7272-41f3-8605-82c365811f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:38.634692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc1ebb-7272-41f3-8605-82c365811f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:38.634885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf3803-f735-4d18-b92f-9211d870921e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:38.635451 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc1ebb-7272-41f3-8605-82c365811f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:38.635565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf3803-f735-4d18-b92f-9211d870921e service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:38.635842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf3803-f735-4d18-b92f-9211d870921e service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:38.636117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf3803-f735-4d18-b92f-9211d870921e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:38.636219 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc1ebb-7272-41f3-8605-82c365811f65 admin admin] PATH_INFO: `/v3/projects/95b570ff2f5e4de3b1567dbb54582086/users/e0e00f8bc62940acab31f0d6d1c82f43/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:38.639103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc1ebb-7272-41f3-8605-82c365811f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b570ff2f5e4de3b1567dbb54582086, user_id=e0e00f8bc62940acab31f0d6d1c82f43, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:38.645147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc1ebb-7272-41f3-8605-82c365811f65 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:38.668448 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 824/1628] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:38 2022] PUT /identity/v3/projects/95b570ff2f5e4de3b1567dbb54582086/users/e0e00f8bc62940acab31f0d6d1c82f43/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:38.679029 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c2e81-a727-43dc-b020-3c865d4b1388 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:38.679552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c2e81-a727-43dc-b020-3c865d4b1388 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:38.679895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c2e81-a727-43dc-b020-3c865d4b1388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:38.727998 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-417c2e81-a727-43dc-b020-3c865d4b1388 None None] MFA Rules not processed for user `e0e00f8bc62940acab31f0d6d1c82f43`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:38.729277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-417c2e81-a727-43dc-b020-3c865d4b1388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:38.735509 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:38.758545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf3803-f735-4d18-b92f-9211d870921e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:38.761541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf3803-f735-4d18-b92f-9211d870921e service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:38.769336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:38.777208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 805/1629] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:32:38.860610 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 825/1630] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:38 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:38.861649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf3803-f735-4d18-b92f-9211d870921e service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:38.873136 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf3803-f735-4d18-b92f-9211d870921e service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:38.965174 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f7266-b049-42be-9d35-f6c600a82e2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:38.966005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f7266-b049-42be-9d35-f6c600a82e2e service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:38.966465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f7266-b049-42be-9d35-f6c600a82e2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:38.966740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f7266-b049-42be-9d35-f6c600a82e2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:39.001581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f7266-b049-42be-9d35-f6c600a82e2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:39.003807 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f7266-b049-42be-9d35-f6c600a82e2e service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:39.019292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 806/1631] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:38 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:39.203946 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c2e81-a727-43dc-b020-3c865d4b1388 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:39.219088 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c2e81-a727-43dc-b020-3c865d4b1388 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:39.236008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c4644-1678-430d-9955-f6edda7a8fa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:39.236789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c4644-1678-430d-9955-f6edda7a8fa2 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:39.237169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c4644-1678-430d-9955-f6edda7a8fa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:39.237519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c4644-1678-430d-9955-f6edda7a8fa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:39.505938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c4644-1678-430d-9955-f6edda7a8fa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:39.507440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c4644-1678-430d-9955-f6edda7a8fa2 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:39.530055 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 826/1632] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:39 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:40.061889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e8bc-eae9-4dd0-8434-6311171b661b None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:40.062954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e8bc-eae9-4dd0-8434-6311171b661b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:40.063435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e8bc-eae9-4dd0-8434-6311171b661b None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:40.065389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 807/1633] 10.222.0.27 () {54 vars in 962 bytes} [Wed Aug 24 12:32:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:40.088098 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8707f-a474-4338-90dc-344046febe62 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:40.089032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8707f-a474-4338-90dc-344046febe62 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:40.089452 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8707f-a474-4338-90dc-344046febe62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:40.123991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-64a8707f-a474-4338-90dc-344046febe62 None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:40.125495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-64a8707f-a474-4338-90dc-344046febe62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:40.348504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 827/1634] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Aug 24 12:32:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:40.354729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef2d2-ce92-4540-9705-67c52c4d9ba6 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:40.355267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef2d2-ce92-4540-9705-67c52c4d9ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:40.355577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef2d2-ce92-4540-9705-67c52c4d9ba6 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:40.357155 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 808/1635] 10.222.0.27 () {54 vars in 912 bytes} [Wed Aug 24 12:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:40.366758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b78d9-9986-4b43-8072-b5a0ad05be45 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:40.368298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b78d9-9986-4b43-8072-b5a0ad05be45 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:40.369128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b78d9-9986-4b43-8072-b5a0ad05be45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:40.400155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6f5b78d9-9986-4b43-8072-b5a0ad05be45 None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:40.402015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6f5b78d9-9986-4b43-8072-b5a0ad05be45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:40.453036 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 828/1636] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:32:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:40.463393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ef2d2-ce92-4540-9705-67c52c4d9ba6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:40.485911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ef2d2-ce92-4540-9705-67c52c4d9ba6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:40.500247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13b5d0-b694-45b6-85bd-d2a16eb6c9c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:40.500801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b5d0-b694-45b6-85bd-d2a16eb6c9c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:40.501083 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b5d0-b694-45b6-85bd-d2a16eb6c9c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:40.501352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b5d0-b694-45b6-85bd-d2a16eb6c9c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:40.551101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13b5d0-b694-45b6-85bd-d2a16eb6c9c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:40.552829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13b5d0-b694-45b6-85bd-d2a16eb6c9c0 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:40.566641 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 809/1637] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:41.053300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490931b3-a120-4e47-9e08-7c999c0bde49 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:41.053983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490931b3-a120-4e47-9e08-7c999c0bde49 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:41.053983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490931b3-a120-4e47-9e08-7c999c0bde49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:41.118590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-490931b3-a120-4e47-9e08-7c999c0bde49 None None] MFA Rules not processed for user `68e836eb8ee7475788d85d2f95329b20`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:41.119838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-490931b3-a120-4e47-9e08-7c999c0bde49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:41.160862 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:41.313203 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 829/1638] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:41 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:41.318781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13b5d0-b694-45b6-85bd-d2a16eb6c9c0 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:41.439449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13b5d0-b694-45b6-85bd-d2a16eb6c9c0 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:41.474494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009f87b-b520-4e37-9209-59f1a00fd7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:41.486690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009f87b-b520-4e37-9209-59f1a00fd7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:41.487224 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009f87b-b520-4e37-9209-59f1a00fd7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:41.487507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009f87b-b520-4e37-9209-59f1a00fd7a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:41.488180 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009f87b-b520-4e37-9209-59f1a00fd7a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:41.499731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009f87b-b520-4e37-9209-59f1a00fd7a3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:41.636097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 810/1639] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:41 2022] POST /identity/v3/projects => generated 405 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:41.637318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490931b3-a120-4e47-9e08-7c999c0bde49 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:41.654481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490931b3-a120-4e47-9e08-7c999c0bde49 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:41.809154 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf53567-7b7d-4eda-9362-dbff6fd9ad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:41.809786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf53567-7b7d-4eda-9362-dbff6fd9ad23 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:41.810065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf53567-7b7d-4eda-9362-dbff6fd9ad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:41.810356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf53567-7b7d-4eda-9362-dbff6fd9ad23 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:41.811009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf53567-7b7d-4eda-9362-dbff6fd9ad23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:41.813205 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf53567-7b7d-4eda-9362-dbff6fd9ad23 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:41.864357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 830/1640] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:41 2022] POST /identity/v3/users => generated 309 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:41.869527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009f87b-b520-4e37-9209-59f1a00fd7a3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:41.966113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009f87b-b520-4e37-9209-59f1a00fd7a3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:41.966113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508e65a-1cc5-4220-8bcf-26d3838d4350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:41.966113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508e65a-1cc5-4220-8bcf-26d3838d4350 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:41.966113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508e65a-1cc5-4220-8bcf-26d3838d4350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:41.968802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508e65a-1cc5-4220-8bcf-26d3838d4350 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:41.968802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508e65a-1cc5-4220-8bcf-26d3838d4350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:41.968802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508e65a-1cc5-4220-8bcf-26d3838d4350 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:41.984943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 811/1641] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:41.989715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf53567-7b7d-4eda-9362-dbff6fd9ad23 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:42.005729 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf53567-7b7d-4eda-9362-dbff6fd9ad23 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:42.019352 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef6585-d578-4ab0-9f08-b4182659249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:42.020173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef6585-d578-4ab0-9f08-b4182659249f admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:42.020328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef6585-d578-4ab0-9f08-b4182659249f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:42.025959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef6585-d578-4ab0-9f08-b4182659249f admin admin] PATH_INFO: `/v3/projects/63688ef320ce4a5889df0ac904ffcd3c/users/a58c68bfe4e04147b77b51051de818c4/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:42.073432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef6585-d578-4ab0-9f08-b4182659249f admin admin] RBAC: Authorizing `identity:create_grant(project_id=63688ef320ce4a5889df0ac904ffcd3c, user_id=a58c68bfe4e04147b77b51051de818c4, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:42.078611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef6585-d578-4ab0-9f08-b4182659249f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:42.125293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 831/1642] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:41 2022] PUT /identity/v3/projects/63688ef320ce4a5889df0ac904ffcd3c/users/a58c68bfe4e04147b77b51051de818c4/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:42.130245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508e65a-1cc5-4220-8bcf-26d3838d4350 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:42.144611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508e65a-1cc5-4220-8bcf-26d3838d4350 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:42.291143 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645380b-e7aa-47bb-a333-51a3e8eaca99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:42.292888 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645380b-e7aa-47bb-a333-51a3e8eaca99 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:42.293372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645380b-e7aa-47bb-a333-51a3e8eaca99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:42.293835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645380b-e7aa-47bb-a333-51a3e8eaca99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:42.294700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645380b-e7aa-47bb-a333-51a3e8eaca99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:42.296575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645380b-e7aa-47bb-a333-51a3e8eaca99 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:42.315534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:42.352311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 812/1643] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:42.355044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef6585-d578-4ab0-9f08-b4182659249f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:42.385338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef6585-d578-4ab0-9f08-b4182659249f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:42.411459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d1bd9-a2d8-4a8e-9b82-60a4d8526300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:42.411950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d1bd9-a2d8-4a8e-9b82-60a4d8526300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:42.412218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d1bd9-a2d8-4a8e-9b82-60a4d8526300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:42.412493 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d1bd9-a2d8-4a8e-9b82-60a4d8526300 admin admin] PATH_INFO: `/v3/projects/63688ef320ce4a5889df0ac904ffcd3c/users/a58c68bfe4e04147b77b51051de818c4/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:42.420678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d1bd9-a2d8-4a8e-9b82-60a4d8526300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63688ef320ce4a5889df0ac904ffcd3c, user_id=a58c68bfe4e04147b77b51051de818c4, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:42.426096 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d1bd9-a2d8-4a8e-9b82-60a4d8526300 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:42.483885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 832/1644] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:42 2022] PUT /identity/v3/projects/63688ef320ce4a5889df0ac904ffcd3c/users/a58c68bfe4e04147b77b51051de818c4/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:46.751775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645380b-e7aa-47bb-a333-51a3e8eaca99 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:46.765069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645380b-e7aa-47bb-a333-51a3e8eaca99 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:46.903848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acceb4f-cd4e-4663-97bc-1f0e4a9ec939 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:46.904838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acceb4f-cd4e-4663-97bc-1f0e4a9ec939 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:46.905505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acceb4f-cd4e-4663-97bc-1f0e4a9ec939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:46.906765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:46.907821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:46.908947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:46.909428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] PATH_INFO: `/v3/users/b71e3ef17a354f0da5357b119b053bcc` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:46.911200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b71e3ef17a354f0da5357b119b053bcc)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:46.913851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:46.978390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0acceb4f-cd4e-4663-97bc-1f0e4a9ec939 None None] MFA Rules not processed for user `a58c68bfe4e04147b77b51051de818c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:47.000084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0acceb4f-cd4e-4663-97bc-1f0e4a9ec939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:47.082275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b71e3ef17a354f0da5357b119b053bcc'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:47.083104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b71e3ef17a354f0da5357b119b053bcc'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:47.085685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b71e3ef17a354f0da5357b119b053bcc'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:47.143854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b71e3ef17a354f0da5357b119b053bcc'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:47.154045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:47.167111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b71e3ef17a354f0da5357b119b053bcc'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:47.175612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 833/1645] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:47.197817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 813/1646] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:46 2022] DELETE /identity/v3/users/b71e3ef17a354f0da5357b119b053bcc => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:47.202350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ece3fb-1823-4398-bbbb-220cf64d76b2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:47.202826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ece3fb-1823-4398-bbbb-220cf64d76b2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:47.203239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ece3fb-1823-4398-bbbb-220cf64d76b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:47.238093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-79ece3fb-1823-4398-bbbb-220cf64d76b2 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:47.239370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-79ece3fb-1823-4398-bbbb-220cf64d76b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:47.388057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 834/1647] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:47.412698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:47.414401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:47.594265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f9554-c20e-4475-bd7e-f591c0b88ac2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:47.594550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f9554-c20e-4475-bd7e-f591c0b88ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:47.594822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f9554-c20e-4475-bd7e-f591c0b88ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:47.631956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0b985-4e69-4307-a030-5cae3d3a6cbd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:47.648236 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40657ae4-29aa-4c1f-80f4-c83c1feded94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:47.651980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40657ae4-29aa-4c1f-80f4-c83c1feded94 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:47.654519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40657ae4-29aa-4c1f-80f4-c83c1feded94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:47.656785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40657ae4-29aa-4c1f-80f4-c83c1feded94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:47.710792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40657ae4-29aa-4c1f-80f4-c83c1feded94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:47.713721 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2b2f9554-c20e-4475-bd7e-f591c0b88ac2 None None] MFA Rules not processed for user `85d8138d44bc40dd8f63ee4916ffbce9`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:47.715212 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2b2f9554-c20e-4475-bd7e-f591c0b88ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:47.717655 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40657ae4-29aa-4c1f-80f4-c83c1feded94 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:47.778211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 814/1648] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:47.829621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40657ae4-29aa-4c1f-80f4-c83c1feded94 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:47.832360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-40657ae4-29aa-4c1f-80f4-c83c1feded94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:47.871426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40657ae4-29aa-4c1f-80f4-c83c1feded94 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:47.892060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aae70-5c3f-49e2-bdfc-ee8dc20d89f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:47.892950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aae70-5c3f-49e2-bdfc-ee8dc20d89f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:47.893248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aae70-5c3f-49e2-bdfc-ee8dc20d89f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:47.896867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aae70-5c3f-49e2-bdfc-ee8dc20d89f9 admin admin] PATH_INFO: `/v3/users/63abae805aae45509764b9747f248ad2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:47.903803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747aae70-5c3f-49e2-bdfc-ee8dc20d89f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63abae805aae45509764b9747f248ad2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:47.910446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747aae70-5c3f-49e2-bdfc-ee8dc20d89f9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:47.995620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:48.030112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 835/1649] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:48.053958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f9554-c20e-4475-bd7e-f591c0b88ac2 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:48.055583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2b2f9554-c20e-4475-bd7e-f591c0b88ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:48.089707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-747aae70-5c3f-49e2-bdfc-ee8dc20d89f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63abae805aae45509764b9747f248ad2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:48.091101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-747aae70-5c3f-49e2-bdfc-ee8dc20d89f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63abae805aae45509764b9747f248ad2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:48.091873 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-747aae70-5c3f-49e2-bdfc-ee8dc20d89f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63abae805aae45509764b9747f248ad2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:48.123987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-747aae70-5c3f-49e2-bdfc-ee8dc20d89f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63abae805aae45509764b9747f248ad2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:48.143476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-747aae70-5c3f-49e2-bdfc-ee8dc20d89f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63abae805aae45509764b9747f248ad2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:48.171728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 815/1650] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:47 2022] DELETE /identity/v3/users/63abae805aae45509764b9747f248ad2 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:48.173635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cb015-ae90-4178-afd4-b1ef2774960c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:48.173924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cb015-ae90-4178-afd4-b1ef2774960c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:48.174253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cb015-ae90-4178-afd4-b1ef2774960c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:48.242734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-191cb015-ae90-4178-afd4-b1ef2774960c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:48.247051 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-191cb015-ae90-4178-afd4-b1ef2774960c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:48.256975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f9554-c20e-4475-bd7e-f591c0b88ac2 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:48.335757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:48.452732 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 816/1651] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:48.459245 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e63a45-de4a-45b4-b9e8-86a427c8a227 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:48.462137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e63a45-de4a-45b4-b9e8-86a427c8a227 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:48.464255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e63a45-de4a-45b4-b9e8-86a427c8a227 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:48.464619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e63a45-de4a-45b4-b9e8-86a427c8a227 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:48.474740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d3e63a45-de4a-45b4-b9e8-86a427c8a227 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:48.497748 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cb015-ae90-4178-afd4-b1ef2774960c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:48.499822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-191cb015-ae90-4178-afd4-b1ef2774960c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:48.605081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cb015-ae90-4178-afd4-b1ef2774960c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:48.612583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc9335-94d8-4dba-922c-3a7c89e225e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:48.614259 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc9335-94d8-4dba-922c-3a7c89e225e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:48.614756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc9335-94d8-4dba-922c-3a7c89e225e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:48.615224 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc9335-94d8-4dba-922c-3a7c89e225e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:48.626676 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc9335-94d8-4dba-922c-3a7c89e225e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:48.629622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc9335-94d8-4dba-922c-3a7c89e225e5 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:48.644577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 817/1652] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:48.671626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e63a45-de4a-45b4-b9e8-86a427c8a227 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:48.673775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e63a45-de4a-45b4-b9e8-86a427c8a227 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:48.701354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 836/1653] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:32:49.489046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc9335-94d8-4dba-922c-3a7c89e225e5 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:49.490627 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-15bc9335-94d8-4dba-922c-3a7c89e225e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:49.523204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc9335-94d8-4dba-922c-3a7c89e225e5 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:49.544425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:49.545851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:49.546289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:49.547224 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] PATH_INFO: `/v3/projects/106a7bed7da74646ab46e21b34d47f10` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:49.550816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=106a7bed7da74646ab46e21b34d47f10)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:49.553835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: WARNING py.warnings [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:32:49.553835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: warnings.warn(msg) Aug 24 12:32:49.553835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: Aug 24 12:32:49.559120 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:49.723070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:49.770155 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '106a7bed7da74646ab46e21b34d47f10', 'request_id': 'req-7f616f1b-15fc-450a-9fb3-ab2cce76d492'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:49.788493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '106a7bed7da74646ab46e21b34d47f10', 'request_id': 'req-7f616f1b-15fc-450a-9fb3-ab2cce76d492'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:49.810442 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '106a7bed7da74646ab46e21b34d47f10', 'request_id': 'req-7f616f1b-15fc-450a-9fb3-ab2cce76d492'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:49.825759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] The token cache is being invalidate because project 106a7bed7da74646ab46e21b34d47f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:49.826557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:49.827528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:49.832829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 818/1654] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:49 2022] DELETE /identity/v3/projects/106a7bed7da74646ab46e21b34d47f10 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:49.837192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e63a45-de4a-45b4-b9e8-86a427c8a227 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:49.855706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e63a45-de4a-45b4-b9e8-86a427c8a227 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:49.993866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86aae3-179f-4278-a95e-b77a196aa88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:49.994929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86aae3-179f-4278-a95e-b77a196aa88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:49.995396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86aae3-179f-4278-a95e-b77a196aa88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:49.995855 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86aae3-179f-4278-a95e-b77a196aa88e admin admin] PATH_INFO: `/v3/users/e0e00f8bc62940acab31f0d6d1c82f43` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:49.998959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86aae3-179f-4278-a95e-b77a196aa88e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0e00f8bc62940acab31f0d6d1c82f43)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:50.001054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: WARNING py.warnings [None req-2c86aae3-179f-4278-a95e-b77a196aa88e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:32:50.001054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: warnings.warn(msg) Aug 24 12:32:50.001054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: Aug 24 12:32:50.005634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86aae3-179f-4278-a95e-b77a196aa88e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:50.146333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2c86aae3-179f-4278-a95e-b77a196aa88e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0e00f8bc62940acab31f0d6d1c82f43'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:50.147740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2c86aae3-179f-4278-a95e-b77a196aa88e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0e00f8bc62940acab31f0d6d1c82f43'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:50.148540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2c86aae3-179f-4278-a95e-b77a196aa88e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0e00f8bc62940acab31f0d6d1c82f43'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:50.176967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:50.181584 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2c86aae3-179f-4278-a95e-b77a196aa88e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0e00f8bc62940acab31f0d6d1c82f43'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:50.201547 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2c86aae3-179f-4278-a95e-b77a196aa88e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0e00f8bc62940acab31f0d6d1c82f43'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:50.227336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 837/1655] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:49 2022] DELETE /identity/v3/users/e0e00f8bc62940acab31f0d6d1c82f43 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:50.754579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:50.756261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:50.782582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f616f1b-15fc-450a-9fb3-ab2cce76d492 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:50.904890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:50.908003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:50.908608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:50.909584 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c admin admin] PATH_INFO: `/v3/projects/27c1d9bc37534d32b9bc0212a80e672b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:50.912497 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=27c1d9bc37534d32b9bc0212a80e672b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:50.915511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:50.951551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9c576-d837-4c0f-af2a-1dc7fbeb6b45 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:50.952365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9c576-d837-4c0f-af2a-1dc7fbeb6b45 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:50.952798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9c576-d837-4c0f-af2a-1dc7fbeb6b45 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:50.955681 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 838/1656] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:32:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:32:51.076999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f445fc-2b0a-45af-a0d5-435ff3a6b4d5 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.077287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f445fc-2b0a-45af-a0d5-435ff3a6b4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.077879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f445fc-2b0a-45af-a0d5-435ff3a6b4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.079480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27c1d9bc37534d32b9bc0212a80e672b', 'request_id': 'req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:51.105290 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27c1d9bc37534d32b9bc0212a80e672b', 'request_id': 'req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:51.111279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-59f445fc-2b0a-45af-a0d5-435ff3a6b4d5 None None] MFA Rules not processed for user `6db648aaa06b476e9a3cafbab832ee68`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:51.114172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-59f445fc-2b0a-45af-a0d5-435ff3a6b4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:51.119517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27c1d9bc37534d32b9bc0212a80e672b', 'request_id': 'req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:51.154131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c admin admin] The token cache is being invalidate because project 27c1d9bc37534d32b9bc0212a80e672b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:51.154131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:51.156523 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9982c62c-fa8d-4e9b-9930-de2de8d17a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:51.164493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 819/1657] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:50 2022] DELETE /identity/v3/projects/27c1d9bc37534d32b9bc0212a80e672b => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:51.166218 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc9ec0-6544-443a-9f76-edfbdc382594 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.166502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc9ec0-6544-443a-9f76-edfbdc382594 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.166776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc9ec0-6544-443a-9f76-edfbdc382594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.211885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 839/1658] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:51.225460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4ecc9ec0-6544-443a-9f76-edfbdc382594 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:51.226888 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4ecc9ec0-6544-443a-9f76-edfbdc382594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:51.272901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73a0df-7a86-439c-ad9d-40fc1f739246 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.273446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73a0df-7a86-439c-ad9d-40fc1f739246 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.273446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73a0df-7a86-439c-ad9d-40fc1f739246 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.275875 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 840/1659] 10.222.0.27 () {54 vars in 961 bytes} [Wed Aug 24 12:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:51.282083 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce76035-8220-4f96-87ac-131a5c53a59c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.282670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce76035-8220-4f96-87ac-131a5c53a59c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.283420 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce76035-8220-4f96-87ac-131a5c53a59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.324056 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:51.334014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8ce76035-8220-4f96-87ac-131a5c53a59c None None] MFA Rules not processed for user `6db648aaa06b476e9a3cafbab832ee68`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:51.338912 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8ce76035-8220-4f96-87ac-131a5c53a59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:51.377976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 820/1660] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:51.382543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc42b1-10ee-45d1-bf9d-2cde7274f5e1 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.382858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc42b1-10ee-45d1-bf9d-2cde7274f5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.383136 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc42b1-10ee-45d1-bf9d-2cde7274f5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.385941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 841/1661] 10.222.0.27 () {58 vars in 1058 bytes} [Wed Aug 24 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:51.387222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc34ece-e651-4b6e-b8a8-3d4713c9c226 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.387519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc34ece-e651-4b6e-b8a8-3d4713c9c226 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.387796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc34ece-e651-4b6e-b8a8-3d4713c9c226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.429105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c4cc42b1-10ee-45d1-bf9d-2cde7274f5e1 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:51.432864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c4cc42b1-10ee-45d1-bf9d-2cde7274f5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:51.450140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9dc34ece-e651-4b6e-b8a8-3d4713c9c226 None None] MFA Rules not processed for user `8e1c70920a8a43999c308c8f595b4528`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:51.451376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9dc34ece-e651-4b6e-b8a8-3d4713c9c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:51.456085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 821/1662] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:51.460721 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc42b1-10ee-45d1-bf9d-2cde7274f5e1 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:51.486776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc42b1-10ee-45d1-bf9d-2cde7274f5e1 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:51.499879 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b4274-050b-4b94-afbb-1eea3d9044bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:51.500502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b4274-050b-4b94-afbb-1eea3d9044bc service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.500811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b4274-050b-4b94-afbb-1eea3d9044bc service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.501543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b4274-050b-4b94-afbb-1eea3d9044bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.512477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b4274-050b-4b94-afbb-1eea3d9044bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:51.515200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b4274-050b-4b94-afbb-1eea3d9044bc service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:51.529637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 822/1663] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:51.530498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b4274-050b-4b94-afbb-1eea3d9044bc service placement] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:51.542393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b4274-050b-4b94-afbb-1eea3d9044bc service placement] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:51.547881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9521e3-3eb2-40e7-aa99-22e924b95795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:51.548553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9521e3-3eb2-40e7-aa99-22e924b95795 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.548867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9521e3-3eb2-40e7-aa99-22e924b95795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.549556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9521e3-3eb2-40e7-aa99-22e924b95795 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.550462 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9521e3-3eb2-40e7-aa99-22e924b95795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:51.552910 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9521e3-3eb2-40e7-aa99-22e924b95795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:51.554914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9521e3-3eb2-40e7-aa99-22e924b95795 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:51.576909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 823/1664] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:51.577867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9521e3-3eb2-40e7-aa99-22e924b95795 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:51.589263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9521e3-3eb2-40e7-aa99-22e924b95795 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:51.598923 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51992856-1b59-469b-8c26-319c911ba38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:51.600128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51992856-1b59-469b-8c26-319c911ba38a admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.600531 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51992856-1b59-469b-8c26-319c911ba38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.601125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51992856-1b59-469b-8c26-319c911ba38a admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.602446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51992856-1b59-469b-8c26-319c911ba38a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:51.604600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 842/1665] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:51.605448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51992856-1b59-469b-8c26-319c911ba38a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:51.618178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd9e94-24dc-4175-82b4-03172e569691 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.618735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd9e94-24dc-4175-82b4-03172e569691 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.619218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd9e94-24dc-4175-82b4-03172e569691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.647696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 824/1666] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:51 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:51.648732 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51992856-1b59-469b-8c26-319c911ba38a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:51.650661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-51992856-1b59-469b-8c26-319c911ba38a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:51.672728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-80bd9e94-24dc-4175-82b4-03172e569691 None None] MFA Rules not processed for user `8e1c70920a8a43999c308c8f595b4528`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:51.674540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-80bd9e94-24dc-4175-82b4-03172e569691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:51.751149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51992856-1b59-469b-8c26-319c911ba38a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:51.760364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:51.761109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.761518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.761792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] PATH_INFO: `/v3/users/d6eddc93f36e4af5ba0a85a420e7a6e9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.763954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6eddc93f36e4af5ba0a85a420e7a6e9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:51.781889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:51.809364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 843/1667] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:51.810441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd9e94-24dc-4175-82b4-03172e569691 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:51.823579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:51.836670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd9e94-24dc-4175-82b4-03172e569691 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:51.845670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d14ece-de37-46c5-a35a-bbecb39cdd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:51.845670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d14ece-de37-46c5-a35a-bbecb39cdd0c admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.847351 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d14ece-de37-46c5-a35a-bbecb39cdd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.847743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d14ece-de37-46c5-a35a-bbecb39cdd0c admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.849648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d14ece-de37-46c5-a35a-bbecb39cdd0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:51.851830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d14ece-de37-46c5-a35a-bbecb39cdd0c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:51.919499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 844/1668] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:51 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:51.920725 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d14ece-de37-46c5-a35a-bbecb39cdd0c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:51.922707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-08d14ece-de37-46c5-a35a-bbecb39cdd0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:51.940048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6eddc93f36e4af5ba0a85a420e7a6e9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:51.943327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6eddc93f36e4af5ba0a85a420e7a6e9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:51.944053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6eddc93f36e4af5ba0a85a420e7a6e9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:51.969960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6eddc93f36e4af5ba0a85a420e7a6e9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.012116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6eddc93f36e4af5ba0a85a420e7a6e9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.040474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 825/1669] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:51 2022] DELETE /identity/v3/users/d6eddc93f36e4af5ba0a85a420e7a6e9 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:52.042018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:52.050546 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.095488 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:52.164027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d14ece-de37-46c5-a35a-bbecb39cdd0c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:52.238157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493c35b-4c87-43bb-9ca6-400b06ea4129 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:52.251703 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dafa22-21b6-40e1-bd2f-c40cab5b6188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:52.252706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dafa22-21b6-40e1-bd2f-c40cab5b6188 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.253306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dafa22-21b6-40e1-bd2f-c40cab5b6188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.253714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dafa22-21b6-40e1-bd2f-c40cab5b6188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.254661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dafa22-21b6-40e1-bd2f-c40cab5b6188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:52.258536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dafa22-21b6-40e1-bd2f-c40cab5b6188 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:52.290035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 826/1670] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:52.293139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c06c93-13d8-4c37-a9f3-a15e6e9a246c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.294518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c06c93-13d8-4c37-a9f3-a15e6e9a246c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.295088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c06c93-13d8-4c37-a9f3-a15e6e9a246c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.328320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-99c06c93-13d8-4c37-a9f3-a15e6e9a246c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:52.329924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-99c06c93-13d8-4c37-a9f3-a15e6e9a246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.342366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 827/1671] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:52.343629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c06c93-13d8-4c37-a9f3-a15e6e9a246c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:52.345271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-99c06c93-13d8-4c37-a9f3-a15e6e9a246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.365395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da83bf8-eebc-4522-8491-ca5b876ea817 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:52.366193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da83bf8-eebc-4522-8491-ca5b876ea817 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.366567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da83bf8-eebc-4522-8491-ca5b876ea817 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.375211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c06c93-13d8-4c37-a9f3-a15e6e9a246c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:52.375836 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da83bf8-eebc-4522-8491-ca5b876ea817 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.382120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4da83bf8-eebc-4522-8491-ca5b876ea817 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.388786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:52.393484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.393942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.394348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] PATH_INFO: `/v3/projects/95b570ff2f5e4de3b1567dbb54582086` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.409817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95b570ff2f5e4de3b1567dbb54582086)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:52.412141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:52.509146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da83bf8-eebc-4522-8491-ca5b876ea817 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:52.509631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95b570ff2f5e4de3b1567dbb54582086', 'request_id': 'req-e7fef17f-7c6b-4950-b80f-204faf496b15'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.511392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da83bf8-eebc-4522-8491-ca5b876ea817 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:52.524332 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 845/1672] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:52.525582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da83bf8-eebc-4522-8491-ca5b876ea817 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:52.538904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da83bf8-eebc-4522-8491-ca5b876ea817 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:52.543876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95b570ff2f5e4de3b1567dbb54582086', 'request_id': 'req-e7fef17f-7c6b-4950-b80f-204faf496b15'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.558123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:52.564274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95b570ff2f5e4de3b1567dbb54582086', 'request_id': 'req-e7fef17f-7c6b-4950-b80f-204faf496b15'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.597547 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] The token cache is being invalidate because project 95b570ff2f5e4de3b1567dbb54582086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:52.598082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.599716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.607339 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 828/1673] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:52 2022] DELETE /identity/v3/projects/95b570ff2f5e4de3b1567dbb54582086 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:52.608437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:52.611662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.649600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a941b22-e585-4008-9f34-7eb6e072f833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:52.650248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a941b22-e585-4008-9f34-7eb6e072f833 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.651089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a941b22-e585-4008-9f34-7eb6e072f833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.651089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a941b22-e585-4008-9f34-7eb6e072f833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.651446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a941b22-e585-4008-9f34-7eb6e072f833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:52.655839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a941b22-e585-4008-9f34-7eb6e072f833 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:52.694076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 846/1674] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:52.704975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d212f6-7094-4cd5-8f42-bd028b69f429 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.705534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d212f6-7094-4cd5-8f42-bd028b69f429 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.705835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d212f6-7094-4cd5-8f42-bd028b69f429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.744354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-54d212f6-7094-4cd5-8f42-bd028b69f429 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:52.747120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-54d212f6-7094-4cd5-8f42-bd028b69f429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.771111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 847/1675] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:52.779779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d212f6-7094-4cd5-8f42-bd028b69f429 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:52.788487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-54d212f6-7094-4cd5-8f42-bd028b69f429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.813712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d212f6-7094-4cd5-8f42-bd028b69f429 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:52.815181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fef17f-7c6b-4950-b80f-204faf496b15 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:52.822346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6055d8-6c28-4f75-9bd0-84e1e17cd3b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:52.823172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6055d8-6c28-4f75-9bd0-84e1e17cd3b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.823660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6055d8-6c28-4f75-9bd0-84e1e17cd3b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.824220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6055d8-6c28-4f75-9bd0-84e1e17cd3b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.826885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3b6055d8-6c28-4f75-9bd0-84e1e17cd3b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.829124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00ca8b-6a4e-4cbf-863c-b7fee7e58261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:52.829864 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00ca8b-6a4e-4cbf-863c-b7fee7e58261 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.830701 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00ca8b-6a4e-4cbf-863c-b7fee7e58261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.831045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00ca8b-6a4e-4cbf-863c-b7fee7e58261 admin admin] PATH_INFO: `/v3/projects/035366777d4148938b22d4f33eccae60/users/3afd2d65e39e4392a8c0c82400f28057/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.843842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6055d8-6c28-4f75-9bd0-84e1e17cd3b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:52.846445 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6055d8-6c28-4f75-9bd0-84e1e17cd3b7 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:52.860285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 829/1676] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:52.863275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886a188-8784-40b9-b47d-a899f5a079e3 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.863725 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886a188-8784-40b9-b47d-a899f5a079e3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.864139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886a188-8784-40b9-b47d-a899f5a079e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.869577 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00ca8b-6a4e-4cbf-863c-b7fee7e58261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=035366777d4148938b22d4f33eccae60, user_id=3afd2d65e39e4392a8c0c82400f28057, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:52.872265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00ca8b-6a4e-4cbf-863c-b7fee7e58261 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:52.890345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e886a188-8784-40b9-b47d-a899f5a079e3 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:52.891878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e886a188-8784-40b9-b47d-a899f5a079e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.902505 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 830/1677] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:52.905252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 848/1678] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:52 2022] PUT /identity/v3/projects/035366777d4148938b22d4f33eccae60/users/3afd2d65e39e4392a8c0c82400f28057/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:52.906743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886a188-8784-40b9-b47d-a899f5a079e3 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:52.920536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00629dba-7fe1-450e-a3f5-ec58e4fb846e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.920979 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00629dba-7fe1-450e-a3f5-ec58e4fb846e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.921100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00629dba-7fe1-450e-a3f5-ec58e4fb846e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.935131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886a188-8784-40b9-b47d-a899f5a079e3 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:52.965437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:52.971911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-00629dba-7fe1-450e-a3f5-ec58e4fb846e None None] MFA Rules not processed for user `3afd2d65e39e4392a8c0c82400f28057`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:52.973174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-00629dba-7fe1-450e-a3f5-ec58e4fb846e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:53.044288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f31de7-aa69-4dfd-8771-7c51bf910890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:53.045406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f31de7-aa69-4dfd-8771-7c51bf910890 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.045810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f31de7-aa69-4dfd-8771-7c51bf910890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.046179 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f31de7-aa69-4dfd-8771-7c51bf910890 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.047425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f31de7-aa69-4dfd-8771-7c51bf910890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:53.048359 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f31de7-aa69-4dfd-8771-7c51bf910890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:53.050319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f31de7-aa69-4dfd-8771-7c51bf910890 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:53.067984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 831/1679] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:53.073755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f31de7-aa69-4dfd-8771-7c51bf910890 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:53.093551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f31de7-aa69-4dfd-8771-7c51bf910890 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:53.099473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 849/1680] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:53.107956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00629dba-7fe1-450e-a3f5-ec58e4fb846e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:53.108077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf34b30-ce79-4063-94d1-83665fc895dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:53.110023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34b30-ce79-4063-94d1-83665fc895dc admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.110666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-00629dba-7fe1-450e-a3f5-ec58e4fb846e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:53.111218 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34b30-ce79-4063-94d1-83665fc895dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.113799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34b30-ce79-4063-94d1-83665fc895dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.114006 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf34b30-ce79-4063-94d1-83665fc895dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:53.119902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf34b30-ce79-4063-94d1-83665fc895dc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:53.240857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 832/1681] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:53 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:53.240983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00629dba-7fe1-450e-a3f5-ec58e4fb846e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:53.241934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf34b30-ce79-4063-94d1-83665fc895dc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:53.253400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:53.342136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48babb-c83f-4122-8a76-ab86d8cccedc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:53.343791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48babb-c83f-4122-8a76-ab86d8cccedc service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.343791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48babb-c83f-4122-8a76-ab86d8cccedc service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.344038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48babb-c83f-4122-8a76-ab86d8cccedc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.419509 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf34b30-ce79-4063-94d1-83665fc895dc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:53.425949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b3aca-fb62-4671-a9d5-080c1a837e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:53.427001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b3aca-fb62-4671-a9d5-080c1a837e78 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.427377 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b3aca-fb62-4671-a9d5-080c1a837e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.427742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b3aca-fb62-4671-a9d5-080c1a837e78 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.428477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b3aca-fb62-4671-a9d5-080c1a837e78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:53.431334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b3aca-fb62-4671-a9d5-080c1a837e78 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:53.464769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48babb-c83f-4122-8a76-ab86d8cccedc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:53.466941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48babb-c83f-4122-8a76-ab86d8cccedc service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:53.484719 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 833/1682] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:53 2022] POST /identity/v3/users => generated 291 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:53.490774 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b3aca-fb62-4671-a9d5-080c1a837e78 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:53.494628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 850/1683] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:53.504756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b3aca-fb62-4671-a9d5-080c1a837e78 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:53.525830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e40e76-7369-4a33-ad42-d419c495c781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:53.527493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e40e76-7369-4a33-ad42-d419c495c781 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.532942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e40e76-7369-4a33-ad42-d419c495c781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.533511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e40e76-7369-4a33-ad42-d419c495c781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.535808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e40e76-7369-4a33-ad42-d419c495c781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:53.538737 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e40e76-7369-4a33-ad42-d419c495c781 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:53.562336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 834/1684] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:53.565409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48babb-c83f-4122-8a76-ab86d8cccedc service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:53.586003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48babb-c83f-4122-8a76-ab86d8cccedc service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:53.604441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9264da-78e3-40ea-91b1-5251438d40c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:53.605121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9264da-78e3-40ea-91b1-5251438d40c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.605395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9264da-78e3-40ea-91b1-5251438d40c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.605651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9264da-78e3-40ea-91b1-5251438d40c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.606660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9264da-78e3-40ea-91b1-5251438d40c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:53.607893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9264da-78e3-40ea-91b1-5251438d40c3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:53.639523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 851/1685] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:53.644176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e40e76-7369-4a33-ad42-d419c495c781 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:53.693861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e40e76-7369-4a33-ad42-d419c495c781 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:53.704248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241eb984-318b-4122-ac71-7b8137750eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:53.704440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241eb984-318b-4122-ac71-7b8137750eef admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.704895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241eb984-318b-4122-ac71-7b8137750eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.705210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241eb984-318b-4122-ac71-7b8137750eef admin admin] PATH_INFO: `/v3/projects/73e1275c927345128c5acb86879ba0a3/users/aef03c6bf284446d886ba1c575752c4a/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.755249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241eb984-318b-4122-ac71-7b8137750eef admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e1275c927345128c5acb86879ba0a3, user_id=aef03c6bf284446d886ba1c575752c4a, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:53.758181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241eb984-318b-4122-ac71-7b8137750eef admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:53.796607 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 835/1686] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:53 2022] PUT /identity/v3/projects/73e1275c927345128c5acb86879ba0a3/users/aef03c6bf284446d886ba1c575752c4a/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:53.807476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1b262-b61c-4e06-8122-cb48ed67e39a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.807808 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1b262-b61c-4e06-8122-cb48ed67e39a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.808027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1b262-b61c-4e06-8122-cb48ed67e39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.868446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f739b00-e6c8-458d-b6a5-e38abae46472 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.869546 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f739b00-e6c8-458d-b6a5-e38abae46472 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.870149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f739b00-e6c8-458d-b6a5-e38abae46472 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.872607 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 836/1687] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:32:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:32:53.873618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-eda1b262-b61c-4e06-8122-cb48ed67e39a None None] MFA Rules not processed for user `aef03c6bf284446d886ba1c575752c4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:53.874880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-eda1b262-b61c-4e06-8122-cb48ed67e39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:53.905308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d2c6b-2e32-4598-ab79-21092a685f21 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.905308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d2c6b-2e32-4598-ab79-21092a685f21 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.905308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d2c6b-2e32-4598-ab79-21092a685f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.960932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4d4d2c6b-2e32-4598-ab79-21092a685f21 None None] MFA Rules not processed for user `6db648aaa06b476e9a3cafbab832ee68`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:53.971920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4d4d2c6b-2e32-4598-ab79-21092a685f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:54.044442 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:54.075206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 837/1688] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:32:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:54.076379 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d2c6b-2e32-4598-ab79-21092a685f21 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:54.083390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 852/1689] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:54.084621 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1b262-b61c-4e06-8122-cb48ed67e39a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:54.087331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-eda1b262-b61c-4e06-8122-cb48ed67e39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:54.181873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:54.231652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d2c6b-2e32-4598-ab79-21092a685f21 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:54.261308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:54.262641 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.263804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.264501 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] PATH_INFO: `/v3/users/3afd2d65e39e4392a8c0c82400f28057` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.266553 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1b262-b61c-4e06-8122-cb48ed67e39a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:54.267574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3afd2d65e39e4392a8c0c82400f28057)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:54.275290 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:54.286438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:54.287651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.288114 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.288791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] PATH_INFO: `/v3/users/68e836eb8ee7475788d85d2f95329b20` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.290273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] RBAC: Authorizing `identity:delete_user(user_id=68e836eb8ee7475788d85d2f95329b20)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:54.292766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:54.378827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3afd2d65e39e4392a8c0c82400f28057'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.378827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3afd2d65e39e4392a8c0c82400f28057'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.379081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3afd2d65e39e4392a8c0c82400f28057'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.415557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3afd2d65e39e4392a8c0c82400f28057'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.416855 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68e836eb8ee7475788d85d2f95329b20'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.419329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68e836eb8ee7475788d85d2f95329b20'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.420303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68e836eb8ee7475788d85d2f95329b20'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.444689 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3afd2d65e39e4392a8c0c82400f28057'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.469510 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68e836eb8ee7475788d85d2f95329b20'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.481373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 838/1690] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:54 2022] DELETE /identity/v3/users/3afd2d65e39e4392a8c0c82400f28057 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:54.483027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:54.485368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:54.492526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68e836eb8ee7475788d85d2f95329b20'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.541179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 853/1691] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:54 2022] DELETE /identity/v3/users/68e836eb8ee7475788d85d2f95329b20 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:54.542044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:54.554360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:54.600967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dbc7c7-361f-415f-ae26-3496a93176c7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:54.625624 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:54.653941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff2998-1754-4efe-889a-21ec6757f4db admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:54.660512 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef836c54-bfea-4344-bc41-3c19d868be05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:54.662533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef836c54-bfea-4344-bc41-3c19d868be05 service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.662831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef836c54-bfea-4344-bc41-3c19d868be05 service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.663106 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef836c54-bfea-4344-bc41-3c19d868be05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.665978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ef836c54-bfea-4344-bc41-3c19d868be05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:54.712747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef836c54-bfea-4344-bc41-3c19d868be05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:54.715351 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef836c54-bfea-4344-bc41-3c19d868be05 service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:54.731433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 854/1692] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:54.733179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882f9e7-4b9f-4d0e-b36f-08343f32e9c3 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.733469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882f9e7-4b9f-4d0e-b36f-08343f32e9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.733746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882f9e7-4b9f-4d0e-b36f-08343f32e9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.776152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e882f9e7-4b9f-4d0e-b36f-08343f32e9c3 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:54.778683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e882f9e7-4b9f-4d0e-b36f-08343f32e9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:54.814326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52f69e-7ffb-4a75-ba8a-ac531888509e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:54.815078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52f69e-7ffb-4a75-ba8a-ac531888509e admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.815359 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52f69e-7ffb-4a75-ba8a-ac531888509e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.815626 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52f69e-7ffb-4a75-ba8a-ac531888509e admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.817644 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52f69e-7ffb-4a75-ba8a-ac531888509e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:54.822678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52f69e-7ffb-4a75-ba8a-ac531888509e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:54.833620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:54.879096 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 855/1693] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:54.883637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658decf1-f73e-41de-b957-8727b8870cce None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.884637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658decf1-f73e-41de-b957-8727b8870cce None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.885235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658decf1-f73e-41de-b957-8727b8870cce None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.886477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 839/1694] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:54 2022] POST /identity/v3/projects => generated 381 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:54.890389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52f69e-7ffb-4a75-ba8a-ac531888509e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:54.890880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 856/1695] 10.222.0.27 () {54 vars in 912 bytes} [Wed Aug 24 12:32:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:54.897660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d8149-e23e-4417-aa9d-f2a76c471daa None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.898071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d8149-e23e-4417-aa9d-f2a76c471daa None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.898413 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d8149-e23e-4417-aa9d-f2a76c471daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.906590 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6f52f69e-7ffb-4a75-ba8a-ac531888509e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:54.994046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c82d8149-e23e-4417-aa9d-f2a76c471daa None None] MFA Rules not processed for user `5e18461c8cc4497eae7a83d9ad1448f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:54.995374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c82d8149-e23e-4417-aa9d-f2a76c471daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.051344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52f69e-7ffb-4a75-ba8a-ac531888509e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.067712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64ae1c-cd13-43ba-8125-fe265c581dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.068358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64ae1c-cd13-43ba-8125-fe265c581dcd admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.068727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64ae1c-cd13-43ba-8125-fe265c581dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.069068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64ae1c-cd13-43ba-8125-fe265c581dcd admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.069787 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb64ae1c-cd13-43ba-8125-fe265c581dcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.073331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb64ae1c-cd13-43ba-8125-fe265c581dcd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.121946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 840/1696] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:54 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:55.121946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64ae1c-cd13-43ba-8125-fe265c581dcd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.122592 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bb64ae1c-cd13-43ba-8125-fe265c581dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.136698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 857/1697] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:32:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:55.137024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64ae1c-cd13-43ba-8125-fe265c581dcd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.139377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d8149-e23e-4417-aa9d-f2a76c471daa None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.143814 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0629f-0f76-4166-a000-ed6249db912c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.144433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0629f-0f76-4166-a000-ed6249db912c service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.144862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0629f-0f76-4166-a000-ed6249db912c service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.145568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0629f-0f76-4166-a000-ed6249db912c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.161056 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0629f-0f76-4166-a000-ed6249db912c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.164063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0629f-0f76-4166-a000-ed6249db912c service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.165965 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d8149-e23e-4417-aa9d-f2a76c471daa None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.177623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 841/1698] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:55.178205 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260081b-b5f9-468b-a617-20de3eb36ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.180174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260081b-b5f9-468b-a617-20de3eb36ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.180474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260081b-b5f9-468b-a617-20de3eb36ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.180771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260081b-b5f9-468b-a617-20de3eb36ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.183328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260081b-b5f9-468b-a617-20de3eb36ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.187800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260081b-b5f9-468b-a617-20de3eb36ca7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.219830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 858/1699] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:55.235222 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0629f-0f76-4166-a000-ed6249db912c service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.266085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0629f-0f76-4166-a000-ed6249db912c service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.269227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711dfb82-358d-452e-98b3-513e047671d4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.269814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711dfb82-358d-452e-98b3-513e047671d4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.270063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711dfb82-358d-452e-98b3-513e047671d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.278530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d0554-efcd-4ead-a48b-f30ff5c956be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.280115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d0554-efcd-4ead-a48b-f30ff5c956be admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.281869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d0554-efcd-4ead-a48b-f30ff5c956be admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.281869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d0554-efcd-4ead-a48b-f30ff5c956be admin admin] PATH_INFO: `/v3/projects/124c68917e2344ec8b73105f70adc44e/users/f957ca52cd2d414d9c9375b423167436/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.330773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-711dfb82-358d-452e-98b3-513e047671d4 None None] MFA Rules not processed for user `b4a617fc13534a89ad50a5fdc7bf769a`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:55.332241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-711dfb82-358d-452e-98b3-513e047671d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.353660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d0554-efcd-4ead-a48b-f30ff5c956be admin admin] RBAC: Authorizing `identity:create_grant(project_id=124c68917e2344ec8b73105f70adc44e, user_id=f957ca52cd2d414d9c9375b423167436, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.360128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d0554-efcd-4ead-a48b-f30ff5c956be admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.406850 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 842/1700] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:55 2022] PUT /identity/v3/projects/124c68917e2344ec8b73105f70adc44e/users/f957ca52cd2d414d9c9375b423167436/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:55.408064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d0554-efcd-4ead-a48b-f30ff5c956be admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.507068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 859/1701] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:55.507809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711dfb82-358d-452e-98b3-513e047671d4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.510294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-711dfb82-358d-452e-98b3-513e047671d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.527172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d0554-efcd-4ead-a48b-f30ff5c956be admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.531419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbf885-4777-46a1-b085-2c7851f9bfef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.532890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbf885-4777-46a1-b085-2c7851f9bfef service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.532890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbf885-4777-46a1-b085-2c7851f9bfef service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.534032 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbf885-4777-46a1-b085-2c7851f9bfef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.540510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:55.543077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711dfb82-358d-452e-98b3-513e047671d4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.554042 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbf885-4777-46a1-b085-2c7851f9bfef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.557452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbf885-4777-46a1-b085-2c7851f9bfef service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.579729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 843/1702] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:55.580774 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbf885-4777-46a1-b085-2c7851f9bfef service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.582248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9bfbf885-4777-46a1-b085-2c7851f9bfef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.631174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:55.649938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.652570 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.652902 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.653340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] PATH_INFO: `/v3/projects/cfeddd326a4045a0a724223bcb0c9487` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.656451 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfeddd326a4045a0a724223bcb0c9487)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.658137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.702996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbf885-4777-46a1-b085-2c7851f9bfef service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.708724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.709660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.710136 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.710605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] PATH_INFO: `/v3/projects/83e6b8558a3146279d84792f251a3dd2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.712043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83e6b8558a3146279d84792f251a3dd2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.714261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.737170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfeddd326a4045a0a724223bcb0c9487', 'request_id': 'req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.747967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfeddd326a4045a0a724223bcb0c9487', 'request_id': 'req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.760611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfeddd326a4045a0a724223bcb0c9487', 'request_id': 'req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.776747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] The token cache is being invalidate because project cfeddd326a4045a0a724223bcb0c9487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:55.777106 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.777931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.781960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 860/1703] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:55 2022] DELETE /identity/v3/projects/cfeddd326a4045a0a724223bcb0c9487 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:55.783644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.784082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83e6b8558a3146279d84792f251a3dd2', 'request_id': 'req-ac19462d-c482-45e0-866c-f216469b1974'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.785585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.798745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83e6b8558a3146279d84792f251a3dd2', 'request_id': 'req-ac19462d-c482-45e0-866c-f216469b1974'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.810130 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83e6b8558a3146279d84792f251a3dd2', 'request_id': 'req-ac19462d-c482-45e0-866c-f216469b1974'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.827248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] The token cache is being invalidate because project 83e6b8558a3146279d84792f251a3dd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:55.828277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.829614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.838280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 844/1704] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:55 2022] DELETE /identity/v3/projects/83e6b8558a3146279d84792f251a3dd2 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:55.839636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.843723 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.978423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5daba-e0c7-40be-ad19-6c9d5f60252e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:56.023978 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19462d-c482-45e0-866c-f216469b1974 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:56.030414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da6a6b-cb45-45d6-bd3e-5dd870a5c2cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:56.031182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da6a6b-cb45-45d6-bd3e-5dd870a5c2cc service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.031461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da6a6b-cb45-45d6-bd3e-5dd870a5c2cc service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.031739 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da6a6b-cb45-45d6-bd3e-5dd870a5c2cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.034029 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a8da6a6b-cb45-45d6-bd3e-5dd870a5c2cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:56.075550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:56.091022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6786758-80c6-4660-9878-c3c3b92e3cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:56.091686 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6786758-80c6-4660-9878-c3c3b92e3cdb admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.091981 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6786758-80c6-4660-9878-c3c3b92e3cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.092423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6786758-80c6-4660-9878-c3c3b92e3cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.093282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6786758-80c6-4660-9878-c3c3b92e3cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:56.095765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6786758-80c6-4660-9878-c3c3b92e3cdb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:56.110732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 861/1705] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:56.112446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a6c35-5455-4044-afcd-a98c79dbb734 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.112726 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a6c35-5455-4044-afcd-a98c79dbb734 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.113146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a6c35-5455-4044-afcd-a98c79dbb734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.139983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-029a6c35-5455-4044-afcd-a98c79dbb734 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:56.142902 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-029a6c35-5455-4044-afcd-a98c79dbb734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:56.153808 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 862/1706] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:56.154929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a6c35-5455-4044-afcd-a98c79dbb734 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:56.156433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-029a6c35-5455-4044-afcd-a98c79dbb734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:56.167934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da6a6b-cb45-45d6-bd3e-5dd870a5c2cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:56.172413 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da6a6b-cb45-45d6-bd3e-5dd870a5c2cc service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:56.173512 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a6c35-5455-4044-afcd-a98c79dbb734 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:56.182539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f2834-b6e2-4c90-a6ae-e9e10d30131b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:56.184337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f2834-b6e2-4c90-a6ae-e9e10d30131b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.184889 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f2834-b6e2-4c90-a6ae-e9e10d30131b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.185483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f2834-b6e2-4c90-a6ae-e9e10d30131b admin admin] PATH_INFO: `/v3/users/a58c68bfe4e04147b77b51051de818c4` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.188240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f2834-b6e2-4c90-a6ae-e9e10d30131b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a58c68bfe4e04147b77b51051de818c4)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:56.191567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f2834-b6e2-4c90-a6ae-e9e10d30131b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:56.207709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 845/1707] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:56.208701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da6a6b-cb45-45d6-bd3e-5dd870a5c2cc service nova] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:56.212412 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a8da6a6b-cb45-45d6-bd3e-5dd870a5c2cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:56.216857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:56.233249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da6a6b-cb45-45d6-bd3e-5dd870a5c2cc service nova] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:56.239462 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8154f-310e-458e-972d-09646d88ad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:56.240303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8154f-310e-458e-972d-09646d88ad2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.240843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8154f-310e-458e-972d-09646d88ad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.241168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8154f-310e-458e-972d-09646d88ad2d admin admin] PATH_INFO: `/v3/projects/124c68917e2344ec8b73105f70adc44e/users/f957ca52cd2d414d9c9375b423167436/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.244524 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8154f-310e-458e-972d-09646d88ad2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=124c68917e2344ec8b73105f70adc44e, user_id=f957ca52cd2d414d9c9375b423167436, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:56.248557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8154f-310e-458e-972d-09646d88ad2d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:56.267237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 846/1708] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:56 2022] PUT /identity/v3/projects/124c68917e2344ec8b73105f70adc44e/users/f957ca52cd2d414d9c9375b423167436/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:56.269381 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcda3d1-70bd-4316-ba21-87e3d312bf39 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.269765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcda3d1-70bd-4316-ba21-87e3d312bf39 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.270699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcda3d1-70bd-4316-ba21-87e3d312bf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.306017 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-0bcda3d1-70bd-4316-ba21-87e3d312bf39 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:56.307624 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0bcda3d1-70bd-4316-ba21-87e3d312bf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:56.313783 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-050f2834-b6e2-4c90-a6ae-e9e10d30131b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a58c68bfe4e04147b77b51051de818c4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:56.318724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-050f2834-b6e2-4c90-a6ae-e9e10d30131b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a58c68bfe4e04147b77b51051de818c4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:56.320201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-050f2834-b6e2-4c90-a6ae-e9e10d30131b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a58c68bfe4e04147b77b51051de818c4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:56.351700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-050f2834-b6e2-4c90-a6ae-e9e10d30131b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a58c68bfe4e04147b77b51051de818c4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:56.382633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-050f2834-b6e2-4c90-a6ae-e9e10d30131b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a58c68bfe4e04147b77b51051de818c4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:56.413120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 863/1709] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:56 2022] DELETE /identity/v3/users/a58c68bfe4e04147b77b51051de818c4 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:56.414289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603c179-2632-4f39-a56a-2d882d7d2fc4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.414606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603c179-2632-4f39-a56a-2d882d7d2fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.414855 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603c179-2632-4f39-a56a-2d882d7d2fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.493803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a603c179-2632-4f39-a56a-2d882d7d2fc4 None None] MFA Rules not processed for user `f957ca52cd2d414d9c9375b423167436`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:56.494711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a603c179-2632-4f39-a56a-2d882d7d2fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:56.542822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 847/1710] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:56.547378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcda3d1-70bd-4316-ba21-87e3d312bf39 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:56.570031 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcda3d1-70bd-4316-ba21-87e3d312bf39 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:56.697111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 864/1711] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:56.699613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603c179-2632-4f39-a56a-2d882d7d2fc4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:56.701408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a603c179-2632-4f39-a56a-2d882d7d2fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:56.836133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413525b-a363-4b84-8d78-5ac5aac02671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:56.841277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413525b-a363-4b84-8d78-5ac5aac02671 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.841633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413525b-a363-4b84-8d78-5ac5aac02671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.841911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413525b-a363-4b84-8d78-5ac5aac02671 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.842625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413525b-a363-4b84-8d78-5ac5aac02671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:56.843210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413525b-a363-4b84-8d78-5ac5aac02671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:56.857777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413525b-a363-4b84-8d78-5ac5aac02671 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:56.868090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603c179-2632-4f39-a56a-2d882d7d2fc4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:56.880163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03318c5b-3ad4-4195-b570-c6ac263ebe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:56.880857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03318c5b-3ad4-4195-b570-c6ac263ebe17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.886288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03318c5b-3ad4-4195-b570-c6ac263ebe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.886609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03318c5b-3ad4-4195-b570-c6ac263ebe17 admin admin] PATH_INFO: `/v3/projects/035366777d4148938b22d4f33eccae60` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.890741 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03318c5b-3ad4-4195-b570-c6ac263ebe17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=035366777d4148938b22d4f33eccae60)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:56.894611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03318c5b-3ad4-4195-b570-c6ac263ebe17 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:56.901897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 848/1712] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:56.920788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413525b-a363-4b84-8d78-5ac5aac02671 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:56.924414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1413525b-a363-4b84-8d78-5ac5aac02671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:56.998900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:57.083626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-03318c5b-3ad4-4195-b570-c6ac263ebe17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '035366777d4148938b22d4f33eccae60', 'request_id': 'req-03318c5b-3ad4-4195-b570-c6ac263ebe17'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:57.083950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:57.108983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-03318c5b-3ad4-4195-b570-c6ac263ebe17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '035366777d4148938b22d4f33eccae60', 'request_id': 'req-03318c5b-3ad4-4195-b570-c6ac263ebe17'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:57.126300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-03318c5b-3ad4-4195-b570-c6ac263ebe17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '035366777d4148938b22d4f33eccae60', 'request_id': 'req-03318c5b-3ad4-4195-b570-c6ac263ebe17'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:57.136760 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413525b-a363-4b84-8d78-5ac5aac02671 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:57.140923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-03318c5b-3ad4-4195-b570-c6ac263ebe17 admin admin] The token cache is being invalidate because project 035366777d4148938b22d4f33eccae60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:57.141225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-03318c5b-3ad4-4195-b570-c6ac263ebe17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:57.141887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-03318c5b-3ad4-4195-b570-c6ac263ebe17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:57.144078 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 865/1713] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:56 2022] DELETE /identity/v3/projects/035366777d4148938b22d4f33eccae60 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:57.145881 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299ebbc-251f-4610-93e8-27836c4a5278 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.145991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299ebbc-251f-4610-93e8-27836c4a5278 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.146220 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299ebbc-251f-4610-93e8-27836c4a5278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.208997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0299ebbc-251f-4610-93e8-27836c4a5278 None None] MFA Rules not processed for user `e34cbcf69b4b4828a3bca077fc8d0694`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:57.209964 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0299ebbc-251f-4610-93e8-27836c4a5278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:57.316832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4d630-8286-4f49-a573-7a62f6cd25e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:57.323383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4d630-8286-4f49-a573-7a62f6cd25e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.324582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4d630-8286-4f49-a573-7a62f6cd25e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.325572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4d630-8286-4f49-a573-7a62f6cd25e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.334651 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e9a4d630-8286-4f49-a573-7a62f6cd25e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:57.338817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 866/1714] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:57.339801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299ebbc-251f-4610-93e8-27836c4a5278 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:57.341537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0299ebbc-251f-4610-93e8-27836c4a5278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:57.594707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4d630-8286-4f49-a573-7a62f6cd25e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:57.596520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4d630-8286-4f49-a573-7a62f6cd25e8 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:57.599076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299ebbc-251f-4610-93e8-27836c4a5278 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:57.614377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd8746-7ff4-4092-95b2-e698e3e9ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:57.615137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd8746-7ff4-4092-95b2-e698e3e9ec3a admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.615572 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd8746-7ff4-4092-95b2-e698e3e9ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.615909 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd8746-7ff4-4092-95b2-e698e3e9ec3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.616955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd8746-7ff4-4092-95b2-e698e3e9ec3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:57.619128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd8746-7ff4-4092-95b2-e698e3e9ec3a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:57.625904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 849/1715] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:57.627364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af9fe1-617f-4f6b-8285-ad9636dd8c40 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.627656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af9fe1-617f-4f6b-8285-ad9636dd8c40 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.627932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af9fe1-617f-4f6b-8285-ad9636dd8c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.653897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-17af9fe1-617f-4f6b-8285-ad9636dd8c40 None None] MFA Rules not processed for user `85d8138d44bc40dd8f63ee4916ffbce9`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:57.655957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-17af9fe1-617f-4f6b-8285-ad9636dd8c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:57.726673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 867/1716] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:57 2022] POST /identity/v3/projects => generated 401 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:57.728412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a6697-192d-4fcc-aaf5-26f860b66094 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.728998 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a6697-192d-4fcc-aaf5-26f860b66094 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.729448 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a6697-192d-4fcc-aaf5-26f860b66094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.793336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 850/1717] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:57.795231 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773b84a-93d4-47dc-b959-c55c5107feb0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.795654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773b84a-93d4-47dc-b959-c55c5107feb0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.796271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773b84a-93d4-47dc-b959-c55c5107feb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.797207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c44a6697-192d-4fcc-aaf5-26f860b66094 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:57.801163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c44a6697-192d-4fcc-aaf5-26f860b66094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:57.832887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7773b84a-93d4-47dc-b959-c55c5107feb0 None None] MFA Rules not processed for user `e34cbcf69b4b4828a3bca077fc8d0694`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:57.834750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7773b84a-93d4-47dc-b959-c55c5107feb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:57.878915 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:57.932700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:57.962868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 851/1718] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:57.967800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773b84a-93d4-47dc-b959-c55c5107feb0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:57.973424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7773b84a-93d4-47dc-b959-c55c5107feb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.037641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 868/1719] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:58.040349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a6697-192d-4fcc-aaf5-26f860b66094 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.044190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c44a6697-192d-4fcc-aaf5-26f860b66094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.060186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a6697-192d-4fcc-aaf5-26f860b66094 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:58.078645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.079776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.080433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.081057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] PATH_INFO: `/v3/projects/63688ef320ce4a5889df0ac904ffcd3c` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.083872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63688ef320ce4a5889df0ac904ffcd3c)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.088138 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.146500 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773b84a-93d4-47dc-b959-c55c5107feb0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:58.158908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f739f8c-4f08-4609-8099-900426495636 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.159677 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f739f8c-4f08-4609-8099-900426495636 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.160018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f739f8c-4f08-4609-8099-900426495636 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.160362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f739f8c-4f08-4609-8099-900426495636 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.165552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2f739f8c-4f08-4609-8099-900426495636 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.188793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f739f8c-4f08-4609-8099-900426495636 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.191275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f739f8c-4f08-4609-8099-900426495636 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.211814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63688ef320ce4a5889df0ac904ffcd3c', 'request_id': 'req-d73f0916-275c-4e84-b216-1a8c571a99c2'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:58.224930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 852/1720] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:58.226273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f739f8c-4f08-4609-8099-900426495636 service placement] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.235079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63688ef320ce4a5889df0ac904ffcd3c', 'request_id': 'req-d73f0916-275c-4e84-b216-1a8c571a99c2'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:58.258653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63688ef320ce4a5889df0ac904ffcd3c', 'request_id': 'req-d73f0916-275c-4e84-b216-1a8c571a99c2'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:58.275127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f739f8c-4f08-4609-8099-900426495636 service placement] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:58.308706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] The token cache is being invalidate because project 63688ef320ce4a5889df0ac904ffcd3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:58.308706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:58.319847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:58.324480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 869/1721] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:58 2022] DELETE /identity/v3/projects/63688ef320ce4a5889df0ac904ffcd3c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:58.332538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.337577 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.411404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827d342-1d34-4149-970d-2ff5b440949f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.412341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827d342-1d34-4149-970d-2ff5b440949f admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.412967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827d342-1d34-4149-970d-2ff5b440949f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.413329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827d342-1d34-4149-970d-2ff5b440949f admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.414067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827d342-1d34-4149-970d-2ff5b440949f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.416204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827d342-1d34-4149-970d-2ff5b440949f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.475291 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 853/1722] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:58 2022] POST /identity/v3/users => generated 301 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:58.476322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827d342-1d34-4149-970d-2ff5b440949f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.487758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5827d342-1d34-4149-970d-2ff5b440949f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.550763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f0916-275c-4e84-b216-1a8c571a99c2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:58.563276 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd347e-fe3b-4eff-a17f-43230318c522 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.567495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd347e-fe3b-4eff-a17f-43230318c522 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.567798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd347e-fe3b-4eff-a17f-43230318c522 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.569672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd347e-fe3b-4eff-a17f-43230318c522 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.574640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-19fd347e-fe3b-4eff-a17f-43230318c522 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.678448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827d342-1d34-4149-970d-2ff5b440949f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:58.691678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183de8e-ba46-47d9-9c99-50aa076e52d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.692566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183de8e-ba46-47d9-9c99-50aa076e52d2 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.693905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183de8e-ba46-47d9-9c99-50aa076e52d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.694592 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183de8e-ba46-47d9-9c99-50aa076e52d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.698997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c183de8e-ba46-47d9-9c99-50aa076e52d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.723250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd347e-fe3b-4eff-a17f-43230318c522 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.727419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd347e-fe3b-4eff-a17f-43230318c522 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.742528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:58.744416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:58.760804 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 870/1723] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:58.763916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33114c-bda5-476d-9682-e27b91dff244 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.764279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33114c-bda5-476d-9682-e27b91dff244 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.764632 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33114c-bda5-476d-9682-e27b91dff244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.815966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-db33114c-bda5-476d-9682-e27b91dff244 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:58.819140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-db33114c-bda5-476d-9682-e27b91dff244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.863983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 871/1724] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:58.872193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db2a6c-586b-41cd-98f1-b392d0658d69 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.872460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db2a6c-586b-41cd-98f1-b392d0658d69 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.872771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db2a6c-586b-41cd-98f1-b392d0658d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.893947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183de8e-ba46-47d9-9c99-50aa076e52d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.900272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183de8e-ba46-47d9-9c99-50aa076e52d2 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.938494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 854/1725] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:58 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:58.949678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e1db2a6c-586b-41cd-98f1-b392d0658d69 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:58.951385 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183de8e-ba46-47d9-9c99-50aa076e52d2 service nova] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.959736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e1db2a6c-586b-41cd-98f1-b392d0658d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.966013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c183de8e-ba46-47d9-9c99-50aa076e52d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:59.005103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 872/1726] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:59.006120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db2a6c-586b-41cd-98f1-b392d0658d69 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.040012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183de8e-ba46-47d9-9c99-50aa076e52d2 service nova] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.044682 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db2a6c-586b-41cd-98f1-b392d0658d69 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.055179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c4d03-f576-46a0-b26b-ad84e7299465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.055817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c4d03-f576-46a0-b26b-ad84e7299465 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.056099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c4d03-f576-46a0-b26b-ad84e7299465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.056373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c4d03-f576-46a0-b26b-ad84e7299465 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.059205 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c4d03-f576-46a0-b26b-ad84e7299465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:59.059521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c4d03-f576-46a0-b26b-ad84e7299465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.060198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3adcfd-372b-48d6-b001-ae1bdc64b2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.062131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3adcfd-372b-48d6-b001-ae1bdc64b2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.063399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c4d03-f576-46a0-b26b-ad84e7299465 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.063514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3adcfd-372b-48d6-b001-ae1bdc64b2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.063911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3adcfd-372b-48d6-b001-ae1bdc64b2b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.077032 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3adcfd-372b-48d6-b001-ae1bdc64b2b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.080982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3adcfd-372b-48d6-b001-ae1bdc64b2b4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.101299 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 873/1727] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:59.103064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062c40e-af33-46c6-8715-fed9be07f735 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.104161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062c40e-af33-46c6-8715-fed9be07f735 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.105270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062c40e-af33-46c6-8715-fed9be07f735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.139233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 855/1728] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:59.139233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3adcfd-372b-48d6-b001-ae1bdc64b2b4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.164403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3adcfd-372b-48d6-b001-ae1bdc64b2b4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.173009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061f92c-4ad8-4336-b9ae-bd6cf2f70f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.174539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061f92c-4ad8-4336-b9ae-bd6cf2f70f9c admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.174969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061f92c-4ad8-4336-b9ae-bd6cf2f70f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.175876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2062c40e-af33-46c6-8715-fed9be07f735 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:59.176535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061f92c-4ad8-4336-b9ae-bd6cf2f70f9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.177205 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2062c40e-af33-46c6-8715-fed9be07f735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:59.177535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061f92c-4ad8-4336-b9ae-bd6cf2f70f9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.180089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061f92c-4ad8-4336-b9ae-bd6cf2f70f9c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.209878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 874/1729] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:59.210838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062c40e-af33-46c6-8715-fed9be07f735 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.233399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062c40e-af33-46c6-8715-fed9be07f735 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.234410 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 856/1730] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:59 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:59.235572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061f92c-4ad8-4336-b9ae-bd6cf2f70f9c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.385110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed00d2-4e2c-4637-aa8c-05c6a9ac2eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.385827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed00d2-4e2c-4637-aa8c-05c6a9ac2eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.386217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed00d2-4e2c-4637-aa8c-05c6a9ac2eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.386595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed00d2-4e2c-4637-aa8c-05c6a9ac2eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.388574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed00d2-4e2c-4637-aa8c-05c6a9ac2eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.390645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed00d2-4e2c-4637-aa8c-05c6a9ac2eb2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.404856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 875/1731] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:59.405919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed00d2-4e2c-4637-aa8c-05c6a9ac2eb2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.421268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed00d2-4e2c-4637-aa8c-05c6a9ac2eb2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.432326 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c8282-864e-4764-ad3c-469945e443f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.433491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c8282-864e-4764-ad3c-469945e443f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.433872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c8282-864e-4764-ad3c-469945e443f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.434291 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c8282-864e-4764-ad3c-469945e443f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.435381 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c8282-864e-4764-ad3c-469945e443f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.437880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c8282-864e-4764-ad3c-469945e443f3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.479958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 876/1732] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:59 2022] POST /identity/v3/users => generated 294 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:59.483095 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061f92c-4ad8-4336-b9ae-bd6cf2f70f9c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.483398 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c8282-864e-4764-ad3c-469945e443f3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.502931 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119057ba-2027-4362-8630-5c3448b39d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.503574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119057ba-2027-4362-8630-5c3448b39d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.503844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119057ba-2027-4362-8630-5c3448b39d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.504116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119057ba-2027-4362-8630-5c3448b39d21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.513131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119057ba-2027-4362-8630-5c3448b39d21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:59.513308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119057ba-2027-4362-8630-5c3448b39d21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.519256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119057ba-2027-4362-8630-5c3448b39d21 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.526861 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c8282-864e-4764-ad3c-469945e443f3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.550565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f5097-05d7-425a-9f76-b8597518b5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.553872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f5097-05d7-425a-9f76-b8597518b5a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.554618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f5097-05d7-425a-9f76-b8597518b5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.554618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f5097-05d7-425a-9f76-b8597518b5a5 admin admin] PATH_INFO: `/v3/projects/c65f948b7eb24ada8decc4b490a2cba0/users/ecd552da44894184adfdbb41831d8c1d/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.567000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 857/1733] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:59.568068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119057ba-2027-4362-8630-5c3448b39d21 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.599667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119057ba-2027-4362-8630-5c3448b39d21 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.620865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fbde8-03dd-442f-80b7-8c7a201bdf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.621491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fbde8-03dd-442f-80b7-8c7a201bdf47 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.621768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fbde8-03dd-442f-80b7-8c7a201bdf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.622039 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fbde8-03dd-442f-80b7-8c7a201bdf47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.624511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fbde8-03dd-442f-80b7-8c7a201bdf47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.624511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fbde8-03dd-442f-80b7-8c7a201bdf47 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.631720 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f5097-05d7-425a-9f76-b8597518b5a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65f948b7eb24ada8decc4b490a2cba0, user_id=ecd552da44894184adfdbb41831d8c1d, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.641424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f5097-05d7-425a-9f76-b8597518b5a5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.699274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 858/1734] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:59.701605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fbde8-03dd-442f-80b7-8c7a201bdf47 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.702765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 877/1735] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:59 2022] PUT /identity/v3/projects/c65f948b7eb24ada8decc4b490a2cba0/users/ecd552da44894184adfdbb41831d8c1d/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:59.705853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f5097-05d7-425a-9f76-b8597518b5a5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.709256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-672fbde8-03dd-442f-80b7-8c7a201bdf47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:59.753022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fbde8-03dd-442f-80b7-8c7a201bdf47 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.888564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f5097-05d7-425a-9f76-b8597518b5a5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.912984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22460571-2ef0-4289-9ec8-070153b53118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.913622 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22460571-2ef0-4289-9ec8-070153b53118 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.913903 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22460571-2ef0-4289-9ec8-070153b53118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.914172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22460571-2ef0-4289-9ec8-070153b53118 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.914808 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22460571-2ef0-4289-9ec8-070153b53118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.916784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22460571-2ef0-4289-9ec8-070153b53118 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.925019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2468e0-70bd-49e0-a9cf-225077fccd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.925589 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2468e0-70bd-49e0-a9cf-225077fccd4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.925867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2468e0-70bd-49e0-a9cf-225077fccd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.926140 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2468e0-70bd-49e0-a9cf-225077fccd4f admin admin] PATH_INFO: `/v3/users/aef03c6bf284446d886ba1c575752c4a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.929543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2468e0-70bd-49e0-a9cf-225077fccd4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef03c6bf284446d886ba1c575752c4a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.931473 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2468e0-70bd-49e0-a9cf-225077fccd4f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.956241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:00.001547 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 878/1736] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:59 2022] POST /identity/v3/projects => generated 357 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:00.005384 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd5cd7-dbfb-492c-a356-23720b326d75 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.005670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd5cd7-dbfb-492c-a356-23720b326d75 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.005942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd5cd7-dbfb-492c-a356-23720b326d75 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.018598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 879/1737] 10.222.0.27 () {54 vars in 912 bytes} [Wed Aug 24 12:33:00 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:33:00.019559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd5cd7-dbfb-492c-a356-23720b326d75 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.059180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:00.119249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ab2468e0-70bd-49e0-a9cf-225077fccd4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef03c6bf284446d886ba1c575752c4a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:00.121384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ab2468e0-70bd-49e0-a9cf-225077fccd4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef03c6bf284446d886ba1c575752c4a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:00.122196 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ab2468e0-70bd-49e0-a9cf-225077fccd4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef03c6bf284446d886ba1c575752c4a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:00.161990 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ab2468e0-70bd-49e0-a9cf-225077fccd4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef03c6bf284446d886ba1c575752c4a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:00.190004 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ab2468e0-70bd-49e0-a9cf-225077fccd4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef03c6bf284446d886ba1c575752c4a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:00.234763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 859/1738] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:59 2022] DELETE /identity/v3/users/aef03c6bf284446d886ba1c575752c4a => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:00.234763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4434a-83c8-4e85-aad7-b230c52b7c29 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.234763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4434a-83c8-4e85-aad7-b230c52b7c29 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.234763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4434a-83c8-4e85-aad7-b230c52b7c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.265969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd5cd7-dbfb-492c-a356-23720b326d75 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.313883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e9a4434a-83c8-4e85-aad7-b230c52b7c29 None None] MFA Rules not processed for user `ecd552da44894184adfdbb41831d8c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:00.317123 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e9a4434a-83c8-4e85-aad7-b230c52b7c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:00.419338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091001c-2713-4ab8-828f-107de08d7485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.420046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091001c-2713-4ab8-828f-107de08d7485 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.420320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091001c-2713-4ab8-828f-107de08d7485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.420584 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091001c-2713-4ab8-828f-107de08d7485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.422687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091001c-2713-4ab8-828f-107de08d7485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.427731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091001c-2713-4ab8-828f-107de08d7485 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.461568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 880/1739] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:00.462553 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091001c-2713-4ab8-828f-107de08d7485 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.467712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8091001c-2713-4ab8-828f-107de08d7485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:00.468228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 860/1740] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:00 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:00.469839 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77773a3e-e5f4-4ab7-8284-dfa41ac2c6bf None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.470122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77773a3e-e5f4-4ab7-8284-dfa41ac2c6bf None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.470750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77773a3e-e5f4-4ab7-8284-dfa41ac2c6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.494811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-77773a3e-e5f4-4ab7-8284-dfa41ac2c6bf None None] MFA Rules not processed for user `5e18461c8cc4497eae7a83d9ad1448f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:00.496206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-77773a3e-e5f4-4ab7-8284-dfa41ac2c6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:00.505583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091001c-2713-4ab8-828f-107de08d7485 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.511187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dc4d3-f3fb-4fac-a26a-fec48ab0c168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.511805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dc4d3-f3fb-4fac-a26a-fec48ab0c168 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.512133 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dc4d3-f3fb-4fac-a26a-fec48ab0c168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.512408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dc4d3-f3fb-4fac-a26a-fec48ab0c168 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.513115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417dc4d3-f3fb-4fac-a26a-fec48ab0c168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.515515 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417dc4d3-f3fb-4fac-a26a-fec48ab0c168 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.556589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 881/1741] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:00 2022] POST /identity/v3/users => generated 279 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:00.558962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc8d8-5018-411a-b0f5-fbd1721b7b72 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.559385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc8d8-5018-411a-b0f5-fbd1721b7b72 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.560005 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc8d8-5018-411a-b0f5-fbd1721b7b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.585706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-53ecc8d8-5018-411a-b0f5-fbd1721b7b72 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:00.587312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-53ecc8d8-5018-411a-b0f5-fbd1721b7b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:00.601980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 882/1742] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:00.603206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc8d8-5018-411a-b0f5-fbd1721b7b72 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.605464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-53ecc8d8-5018-411a-b0f5-fbd1721b7b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:00.621690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc8d8-5018-411a-b0f5-fbd1721b7b72 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.628946 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ecb47-bbbc-4e1f-b885-bda837c427fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.629244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 861/1743] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:33:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:00.630148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ecb47-bbbc-4e1f-b885-bda837c427fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.630435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ecb47-bbbc-4e1f-b885-bda837c427fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.630669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77773a3e-e5f4-4ab7-8284-dfa41ac2c6bf None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.630814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ecb47-bbbc-4e1f-b885-bda837c427fd admin admin] PATH_INFO: `/v3/projects/0f046f87485a425ea099faccf8e3e1c5/users/a7eeccf4d26b4030a72695e8f4b5f629/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.632719 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-77773a3e-e5f4-4ab7-8284-dfa41ac2c6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:00.651622 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ecb47-bbbc-4e1f-b885-bda837c427fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f046f87485a425ea099faccf8e3e1c5, user_id=a7eeccf4d26b4030a72695e8f4b5f629, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.656159 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ecb47-bbbc-4e1f-b885-bda837c427fd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.662506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77773a3e-e5f4-4ab7-8284-dfa41ac2c6bf None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.669748 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7da01-bc7b-4393-ad9f-83d1ecb8eb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.670479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7da01-bc7b-4393-ad9f-83d1ecb8eb45 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.670916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7da01-bc7b-4393-ad9f-83d1ecb8eb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.671514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7da01-bc7b-4393-ad9f-83d1ecb8eb45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.676818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7da01-bc7b-4393-ad9f-83d1ecb8eb45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.682226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7da01-bc7b-4393-ad9f-83d1ecb8eb45 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.684575 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 883/1744] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:00 2022] PUT /identity/v3/projects/0f046f87485a425ea099faccf8e3e1c5/users/a7eeccf4d26b4030a72695e8f4b5f629/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:00.694504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ecb47-bbbc-4e1f-b885-bda837c427fd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.762645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:00.782698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 862/1745] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:00 2022] POST /identity/v3/projects => generated 405 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:00.789788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7da01-bc7b-4393-ad9f-83d1ecb8eb45 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.798590 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-def7da01-bc7b-4393-ad9f-83d1ecb8eb45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:00.867639 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ecb47-bbbc-4e1f-b885-bda837c427fd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.878112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:00.950214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7da01-bc7b-4393-ad9f-83d1ecb8eb45 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.960522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa073758-2cf5-48fa-b5bc-d5797cf1eb36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.964834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa073758-2cf5-48fa-b5bc-d5797cf1eb36 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.966933 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa073758-2cf5-48fa-b5bc-d5797cf1eb36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.967940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa073758-2cf5-48fa-b5bc-d5797cf1eb36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.997485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb9350-ba9d-40de-8617-e8b788a16ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.999793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb9350-ba9d-40de-8617-e8b788a16ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.000797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb9350-ba9d-40de-8617-e8b788a16ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.001919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb9350-ba9d-40de-8617-e8b788a16ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.005075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb9350-ba9d-40de-8617-e8b788a16ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.006962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb9350-ba9d-40de-8617-e8b788a16ef9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.031700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 884/1746] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:01.038916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb9350-ba9d-40de-8617-e8b788a16ef9 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:01.088017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb9350-ba9d-40de-8617-e8b788a16ef9 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:01.102392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71921278-5761-407b-9ba6-4b8fa5be19ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:01.103424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71921278-5761-407b-9ba6-4b8fa5be19ea service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.103930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71921278-5761-407b-9ba6-4b8fa5be19ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.104392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71921278-5761-407b-9ba6-4b8fa5be19ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.118744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71921278-5761-407b-9ba6-4b8fa5be19ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.126295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71921278-5761-407b-9ba6-4b8fa5be19ea service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.188056 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa073758-2cf5-48fa-b5bc-d5797cf1eb36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.230120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 885/1747] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:01.238667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa073758-2cf5-48fa-b5bc-d5797cf1eb36 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.242213 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678cd8b-f472-476e-8447-da53a1134d8c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.245424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678cd8b-f472-476e-8447-da53a1134d8c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.252038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678cd8b-f472-476e-8447-da53a1134d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.311573 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 863/1748] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:01.312703 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa073758-2cf5-48fa-b5bc-d5797cf1eb36 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:01.369700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa073758-2cf5-48fa-b5bc-d5797cf1eb36 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:01.394111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca7196-d6de-4e78-8149-92376573927e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:01.396988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca7196-d6de-4e78-8149-92376573927e admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.399932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2678cd8b-f472-476e-8447-da53a1134d8c None None] MFA Rules not processed for user `a7eeccf4d26b4030a72695e8f4b5f629`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:01.400411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca7196-d6de-4e78-8149-92376573927e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.402674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca7196-d6de-4e78-8149-92376573927e admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.410866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2678cd8b-f472-476e-8447-da53a1134d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:01.413795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca7196-d6de-4e78-8149-92376573927e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.443956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca7196-d6de-4e78-8149-92376573927e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.517260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 864/1749] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:01 2022] POST /identity/v3/users => generated 309 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:01.518467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca7196-d6de-4e78-8149-92376573927e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:01.554757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca7196-d6de-4e78-8149-92376573927e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:01.564400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 886/1750] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:01.567099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27bc3b-9aef-43ba-8d60-f642ffeedc5c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.567513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e162686-e75c-474b-9af5-419969581aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:01.567905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27bc3b-9aef-43ba-8d60-f642ffeedc5c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.568223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e162686-e75c-474b-9af5-419969581aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.568502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e162686-e75c-474b-9af5-419969581aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.568810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e162686-e75c-474b-9af5-419969581aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.569009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27bc3b-9aef-43ba-8d60-f642ffeedc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.573715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e162686-e75c-474b-9af5-419969581aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.575665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e162686-e75c-474b-9af5-419969581aaf admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.600931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-fa27bc3b-9aef-43ba-8d60-f642ffeedc5c None None] MFA Rules not processed for user `b4a617fc13534a89ad50a5fdc7bf769a`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:01.603841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fa27bc3b-9aef-43ba-8d60-f642ffeedc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:01.605025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 865/1751] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:01.605976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e162686-e75c-474b-9af5-419969581aaf admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:01.623388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e162686-e75c-474b-9af5-419969581aaf admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:01.678217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ce343-9db3-47af-bff7-ec9c209c59d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:01.683832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ce343-9db3-47af-bff7-ec9c209c59d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.683986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ce343-9db3-47af-bff7-ec9c209c59d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.684661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ce343-9db3-47af-bff7-ec9c209c59d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.684999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ce343-9db3-47af-bff7-ec9c209c59d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.693588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ce343-9db3-47af-bff7-ec9c209c59d9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.754296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 866/1752] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:01.757706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ce343-9db3-47af-bff7-ec9c209c59d9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:01.780548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ce343-9db3-47af-bff7-ec9c209c59d9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:01.792286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1572329-aaf4-4855-be49-7e9b1bebe8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:01.792944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1572329-aaf4-4855-be49-7e9b1bebe8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.793215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1572329-aaf4-4855-be49-7e9b1bebe8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.793480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1572329-aaf4-4855-be49-7e9b1bebe8d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.794115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1572329-aaf4-4855-be49-7e9b1bebe8d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.795873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 887/1753] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:01.798633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27bc3b-9aef-43ba-8d60-f642ffeedc5c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:01.801511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1572329-aaf4-4855-be49-7e9b1bebe8d6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.826104 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27bc3b-9aef-43ba-8d60-f642ffeedc5c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:01.842374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f642c7-720b-44b8-b6fe-1aeb1c048cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:01.844642 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f642c7-720b-44b8-b6fe-1aeb1c048cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.846581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f642c7-720b-44b8-b6fe-1aeb1c048cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.848620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f642c7-720b-44b8-b6fe-1aeb1c048cb5 admin admin] PATH_INFO: `/v3/projects/15c5ccea4f69476d8b147b8eef356470/users/c6cac6d8b2b9453a879145131ccfc7b6/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.875753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 867/1754] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:01 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:01.876720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1572329-aaf4-4855-be49-7e9b1bebe8d6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:01.900414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f642c7-720b-44b8-b6fe-1aeb1c048cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c5ccea4f69476d8b147b8eef356470, user_id=c6cac6d8b2b9453a879145131ccfc7b6, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.912611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f642c7-720b-44b8-b6fe-1aeb1c048cb5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.954583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 888/1755] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:01 2022] PUT /identity/v3/projects/15c5ccea4f69476d8b147b8eef356470/users/c6cac6d8b2b9453a879145131ccfc7b6/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:01.956700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f642c7-720b-44b8-b6fe-1aeb1c048cb5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:02.006390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:02.052436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:02.083917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1572329-aaf4-4855-be49-7e9b1bebe8d6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:02.200822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e607eb6-0a47-4821-903f-1a80de9c3bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:02.201989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e607eb6-0a47-4821-903f-1a80de9c3bda admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.202265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e607eb6-0a47-4821-903f-1a80de9c3bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.202529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e607eb6-0a47-4821-903f-1a80de9c3bda admin admin] PATH_INFO: `/v3/projects/a3011f05ab064ce998a1a86c9a3ad884/users/8bf7713683bb4acba27510a16568eae0/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.223363 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e607eb6-0a47-4821-903f-1a80de9c3bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3011f05ab064ce998a1a86c9a3ad884, user_id=8bf7713683bb4acba27510a16568eae0, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:02.226672 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e607eb6-0a47-4821-903f-1a80de9c3bda admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:02.248709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 868/1756] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:01 2022] PUT /identity/v3/projects/a3011f05ab064ce998a1a86c9a3ad884/users/8bf7713683bb4acba27510a16568eae0/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:02.250838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61726a7a-658e-43e3-b2e2-87ef1a71a8e0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.251134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61726a7a-658e-43e3-b2e2-87ef1a71a8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.251405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61726a7a-658e-43e3-b2e2-87ef1a71a8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.287007 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-61726a7a-658e-43e3-b2e2-87ef1a71a8e0 None None] MFA Rules not processed for user `c6cac6d8b2b9453a879145131ccfc7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:02.288260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-61726a7a-658e-43e3-b2e2-87ef1a71a8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:02.293521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f642c7-720b-44b8-b6fe-1aeb1c048cb5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:02.377917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 869/1757] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:02.377917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61726a7a-658e-43e3-b2e2-87ef1a71a8e0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:02.478350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835f9a3-e91c-4936-b7fe-640e82f3192c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:02.479246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835f9a3-e91c-4936-b7fe-640e82f3192c admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.479246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835f9a3-e91c-4936-b7fe-640e82f3192c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.479437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835f9a3-e91c-4936-b7fe-640e82f3192c admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.480012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6835f9a3-e91c-4936-b7fe-640e82f3192c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:02.486468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6835f9a3-e91c-4936-b7fe-640e82f3192c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:02.539840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61726a7a-658e-43e3-b2e2-87ef1a71a8e0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:02.547140 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb36cd-1dec-43bd-9af7-a77452180c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:02.547741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb36cd-1dec-43bd-9af7-a77452180c87 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.548064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb36cd-1dec-43bd-9af7-a77452180c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.548343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb36cd-1dec-43bd-9af7-a77452180c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.549250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb36cd-1dec-43bd-9af7-a77452180c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:02.551866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb36cd-1dec-43bd-9af7-a77452180c87 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:02.566146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 889/1758] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:01 2022] POST /identity/v3/users => generated 299 bytes in 610 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:02.567752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835f9a3-e91c-4936-b7fe-640e82f3192c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:02.572496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 870/1759] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:02.573494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb36cd-1dec-43bd-9af7-a77452180c87 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:02.575687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-05cb36cd-1dec-43bd-9af7-a77452180c87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:02.581719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835f9a3-e91c-4936-b7fe-640e82f3192c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:02.588117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c76062-bdf3-442f-bfef-a94b2ef26a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:02.588694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:02.590293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c76062-bdf3-442f-bfef-a94b2ef26a34 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.590889 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c76062-bdf3-442f-bfef-a94b2ef26a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.591365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c76062-bdf3-442f-bfef-a94b2ef26a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.592286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c76062-bdf3-442f-bfef-a94b2ef26a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:02.594010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb36cd-1dec-43bd-9af7-a77452180c87 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:02.596865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c76062-bdf3-442f-bfef-a94b2ef26a34 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:02.601260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:02.601964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.603203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.603493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] PATH_INFO: `/v3/projects/73e1275c927345128c5acb86879ba0a3` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.606176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e1275c927345128c5acb86879ba0a3)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:02.621388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:02.684740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 890/1760] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:02 2022] POST /identity/v3/projects => generated 355 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:02.685966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c76062-bdf3-442f-bfef-a94b2ef26a34 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:02.699099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e1275c927345128c5acb86879ba0a3', 'request_id': 'req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:02.757642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e1275c927345128c5acb86879ba0a3', 'request_id': 'req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:02.778493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e1275c927345128c5acb86879ba0a3', 'request_id': 'req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:02.846187 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] The token cache is being invalidate because project 73e1275c927345128c5acb86879ba0a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:02.852490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:02.854613 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:02.858908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 871/1761] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:02 2022] DELETE /identity/v3/projects/73e1275c927345128c5acb86879ba0a3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:02.865297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:02.870970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:02.884962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c76062-bdf3-442f-bfef-a94b2ef26a34 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:02.904062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7aa023-0e66-46f1-bf75-f9f3cdea626b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:02.904959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7aa023-0e66-46f1-bf75-f9f3cdea626b admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.905065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7aa023-0e66-46f1-bf75-f9f3cdea626b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.905306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7aa023-0e66-46f1-bf75-f9f3cdea626b admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.906107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7aa023-0e66-46f1-bf75-f9f3cdea626b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:02.909900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7aa023-0e66-46f1-bf75-f9f3cdea626b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:02.925035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 891/1762] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:02.925035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b50056-dba6-49c3-a1c9-7dece8d99bd6 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.925035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b50056-dba6-49c3-a1c9-7dece8d99bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.925035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b50056-dba6-49c3-a1c9-7dece8d99bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.999280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-57b50056-dba6-49c3-a1c9-7dece8d99bd6 None None] MFA Rules not processed for user `e34cbcf69b4b4828a3bca077fc8d0694`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:03.032444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-57b50056-dba6-49c3-a1c9-7dece8d99bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:03.079381 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:03.112222 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32bada-1a3e-4cbd-917c-c18cea0aacf1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.121228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0ba65-bef9-48d8-806c-0fa992f566a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.122171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0ba65-bef9-48d8-806c-0fa992f566a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.122782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0ba65-bef9-48d8-806c-0fa992f566a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.123300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0ba65-bef9-48d8-806c-0fa992f566a3 admin admin] PATH_INFO: `/v3/projects/a3011f05ab064ce998a1a86c9a3ad884/users/8bf7713683bb4acba27510a16568eae0/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.125609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0ba65-bef9-48d8-806c-0fa992f566a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3011f05ab064ce998a1a86c9a3ad884, user_id=8bf7713683bb4acba27510a16568eae0, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.129178 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0ba65-bef9-48d8-806c-0fa992f566a3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.174374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 872/1763] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:02 2022] PUT /identity/v3/projects/a3011f05ab064ce998a1a86c9a3ad884/users/8bf7713683bb4acba27510a16568eae0/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:03.186296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0ba65-bef9-48d8-806c-0fa992f566a3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.192713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-28c0ba65-bef9-48d8-806c-0fa992f566a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:03.233045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 892/1764] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:03.238264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b50056-dba6-49c3-a1c9-7dece8d99bd6 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.241730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-57b50056-dba6-49c3-a1c9-7dece8d99bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:03.272980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b50056-dba6-49c3-a1c9-7dece8d99bd6 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.341247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0ba65-bef9-48d8-806c-0fa992f566a3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.389341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec50a2-731e-49a3-8c3c-0eec8a5045dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.392266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc696ff0-583c-44bc-8398-24840a686539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.393179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc696ff0-583c-44bc-8398-24840a686539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.393557 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc696ff0-583c-44bc-8398-24840a686539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.393929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc696ff0-583c-44bc-8398-24840a686539 admin admin] PATH_INFO: `/v3/users/f957ca52cd2d414d9c9375b423167436` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.397334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec50a2-731e-49a3-8c3c-0eec8a5045dc admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.397795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec50a2-731e-49a3-8c3c-0eec8a5045dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.398291 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc696ff0-583c-44bc-8398-24840a686539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f957ca52cd2d414d9c9375b423167436)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.398716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec50a2-731e-49a3-8c3c-0eec8a5045dc admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.400445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc696ff0-583c-44bc-8398-24840a686539 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.401830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec50a2-731e-49a3-8c3c-0eec8a5045dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.421466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec50a2-731e-49a3-8c3c-0eec8a5045dc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.504700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 873/1765] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:03 2022] POST /identity/v3/users => generated 284 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:03.509498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec50a2-731e-49a3-8c3c-0eec8a5045dc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.518505 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-50ec50a2-731e-49a3-8c3c-0eec8a5045dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:03.523513 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fc696ff0-583c-44bc-8398-24840a686539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f957ca52cd2d414d9c9375b423167436'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:03.526631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fc696ff0-583c-44bc-8398-24840a686539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f957ca52cd2d414d9c9375b423167436'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:03.527808 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fc696ff0-583c-44bc-8398-24840a686539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f957ca52cd2d414d9c9375b423167436'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:03.557737 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec50a2-731e-49a3-8c3c-0eec8a5045dc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.567310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fc696ff0-583c-44bc-8398-24840a686539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f957ca52cd2d414d9c9375b423167436'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:03.572604 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de2558-57ac-412c-b5a3-02a614a9d497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.573338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de2558-57ac-412c-b5a3-02a614a9d497 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.573617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de2558-57ac-412c-b5a3-02a614a9d497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.573893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de2558-57ac-412c-b5a3-02a614a9d497 admin admin] PATH_INFO: `/v3/projects/b8db1e553f8a46c9a27c8900ac394946/users/086d2324599348e5a1ece04315109289/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.591080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fc696ff0-583c-44bc-8398-24840a686539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f957ca52cd2d414d9c9375b423167436'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:03.607688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:03.648356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 893/1766] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:03 2022] DELETE /identity/v3/users/f957ca52cd2d414d9c9375b423167436 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:03.649274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de2558-57ac-412c-b5a3-02a614a9d497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8db1e553f8a46c9a27c8900ac394946, user_id=086d2324599348e5a1ece04315109289, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.652310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0678094-315e-48a9-9686-a817e9228f14 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.652602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0678094-315e-48a9-9686-a817e9228f14 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.652901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0678094-315e-48a9-9686-a817e9228f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.661503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de2558-57ac-412c-b5a3-02a614a9d497 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.696447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 874/1767] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:03 2022] PUT /identity/v3/projects/b8db1e553f8a46c9a27c8900ac394946/users/086d2324599348e5a1ece04315109289/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:03.703823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64738ec6-e5e0-4a40-8ffc-c376d5491abb None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.705004 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64738ec6-e5e0-4a40-8ffc-c376d5491abb None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.705389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64738ec6-e5e0-4a40-8ffc-c376d5491abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.826298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-64738ec6-e5e0-4a40-8ffc-c376d5491abb None None] MFA Rules not processed for user `8bf7713683bb4acba27510a16568eae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:03.831745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-64738ec6-e5e0-4a40-8ffc-c376d5491abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:03.845105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e0678094-315e-48a9-9686-a817e9228f14 None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:03.848575 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e0678094-315e-48a9-9686-a817e9228f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:03.880292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:04.013558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 875/1768] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:04.013558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64738ec6-e5e0-4a40-8ffc-c376d5491abb None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:04.013558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-64738ec6-e5e0-4a40-8ffc-c376d5491abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:04.043348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 894/1769] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:04.048540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0678094-315e-48a9-9686-a817e9228f14 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:04.053580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e0678094-315e-48a9-9686-a817e9228f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:04.188918 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64738ec6-e5e0-4a40-8ffc-c376d5491abb None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:04.204570 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a711ea-df03-465d-afc5-cf1435f798e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:04.205907 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a711ea-df03-465d-afc5-cf1435f798e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:04.206192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a711ea-df03-465d-afc5-cf1435f798e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:04.206465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a711ea-df03-465d-afc5-cf1435f798e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:04.207132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a711ea-df03-465d-afc5-cf1435f798e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:04.208818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a711ea-df03-465d-afc5-cf1435f798e6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:04.243225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0678094-315e-48a9-9686-a817e9228f14 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:04.244852 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 876/1770] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:04.248307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a711ea-df03-465d-afc5-cf1435f798e6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:04.253480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f2a711ea-df03-465d-afc5-cf1435f798e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:04.256849 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912b2e1-2774-4184-9673-08024f44ab8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:04.258658 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912b2e1-2774-4184-9673-08024f44ab8d admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:04.258943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912b2e1-2774-4184-9673-08024f44ab8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:04.259238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912b2e1-2774-4184-9673-08024f44ab8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:04.259886 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912b2e1-2774-4184-9673-08024f44ab8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:04.261587 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912b2e1-2774-4184-9673-08024f44ab8d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:04.297038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 895/1771] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:04.303603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912b2e1-2774-4184-9673-08024f44ab8d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:04.344827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912b2e1-2774-4184-9673-08024f44ab8d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:04.351954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e9520-d825-44f6-b752-f180b6c65b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:04.352510 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e9520-d825-44f6-b752-f180b6c65b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:04.352838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e9520-d825-44f6-b752-f180b6c65b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:04.353094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e9520-d825-44f6-b752-f180b6c65b26 admin admin] PATH_INFO: `/v3/projects/88150ecec31e4f4d9a6767523a8571f2/users/ab4a7429267a4b70a4a30c8647fd5437/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:04.373283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e9520-d825-44f6-b752-f180b6c65b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88150ecec31e4f4d9a6767523a8571f2, user_id=ab4a7429267a4b70a4a30c8647fd5437, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:04.377317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e9520-d825-44f6-b752-f180b6c65b26 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:04.401353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 896/1772] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:04 2022] PUT /identity/v3/projects/88150ecec31e4f4d9a6767523a8571f2/users/ab4a7429267a4b70a4a30c8647fd5437/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:04.405992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e9520-d825-44f6-b752-f180b6c65b26 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:04.433909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a711ea-df03-465d-afc5-cf1435f798e6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:04.475294 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:04.524913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e9520-d825-44f6-b752-f180b6c65b26 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:04.532623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58748f74-d5da-4104-8323-e7ee54aa3778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:04.533875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4525c1-3c37-4c68-93a2-1161d157bf61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:04.534926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4525c1-3c37-4c68-93a2-1161d157bf61 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:04.535266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4525c1-3c37-4c68-93a2-1161d157bf61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:04.535541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4525c1-3c37-4c68-93a2-1161d157bf61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:04.537005 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58748f74-d5da-4104-8323-e7ee54aa3778 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:04.537601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58748f74-d5da-4104-8323-e7ee54aa3778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:04.538020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58748f74-d5da-4104-8323-e7ee54aa3778 admin admin] PATH_INFO: `/v3/projects/b8db1e553f8a46c9a27c8900ac394946/users/086d2324599348e5a1ece04315109289/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:04.544057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58748f74-d5da-4104-8323-e7ee54aa3778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8db1e553f8a46c9a27c8900ac394946, user_id=086d2324599348e5a1ece04315109289, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:04.555926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58748f74-d5da-4104-8323-e7ee54aa3778 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:04.603366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 897/1773] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:04 2022] PUT /identity/v3/projects/b8db1e553f8a46c9a27c8900ac394946/users/086d2324599348e5a1ece04315109289/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:04.605100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58748f74-d5da-4104-8323-e7ee54aa3778 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:04.606865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-58748f74-d5da-4104-8323-e7ee54aa3778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:04.639656 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:04.694168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4525c1-3c37-4c68-93a2-1161d157bf61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:04.695659 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4525c1-3c37-4c68-93a2-1161d157bf61 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:04.710633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 877/1774] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:04 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:04.711632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4525c1-3c37-4c68-93a2-1161d157bf61 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:04.733917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4525c1-3c37-4c68-93a2-1161d157bf61 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:04.822415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58748f74-d5da-4104-8323-e7ee54aa3778 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:04.831588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227f3bb-ae6b-40c5-bb05-6bcfb5e73e09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:04.833052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227f3bb-ae6b-40c5-bb05-6bcfb5e73e09 service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:04.833358 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227f3bb-ae6b-40c5-bb05-6bcfb5e73e09 service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:04.833690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227f3bb-ae6b-40c5-bb05-6bcfb5e73e09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:04.908149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fddd5-66ef-46a1-93e8-b5bcda97349f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:04.908808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fddd5-66ef-46a1-93e8-b5bcda97349f admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:04.909163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fddd5-66ef-46a1-93e8-b5bcda97349f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:04.909346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fddd5-66ef-46a1-93e8-b5bcda97349f admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:04.910009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fddd5-66ef-46a1-93e8-b5bcda97349f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:04.911671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fddd5-66ef-46a1-93e8-b5bcda97349f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:04.950679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 878/1775] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:04.950679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad24d-ea49-4b90-b0c7-58f42b5c0392 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:04.950679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad24d-ea49-4b90-b0c7-58f42b5c0392 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:04.950679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad24d-ea49-4b90-b0c7-58f42b5c0392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:05.009811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ea9ad24d-ea49-4b90-b0c7-58f42b5c0392 None None] MFA Rules not processed for user `086d2324599348e5a1ece04315109289`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:05.012094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ea9ad24d-ea49-4b90-b0c7-58f42b5c0392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:05.038680 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227f3bb-ae6b-40c5-bb05-6bcfb5e73e09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:05.040342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227f3bb-ae6b-40c5-bb05-6bcfb5e73e09 service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:05.064167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 898/1776] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:33:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:33:05.079641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227f3bb-ae6b-40c5-bb05-6bcfb5e73e09 service swift] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:05.112192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227f3bb-ae6b-40c5-bb05-6bcfb5e73e09 service swift] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:05.141533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb756f28-5eec-4698-be87-af56b4b1cf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:05.142323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb756f28-5eec-4698-be87-af56b4b1cf54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:05.142606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb756f28-5eec-4698-be87-af56b4b1cf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:05.143289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb756f28-5eec-4698-be87-af56b4b1cf54 admin admin] PATH_INFO: `/v3/projects/88150ecec31e4f4d9a6767523a8571f2/users/ab4a7429267a4b70a4a30c8647fd5437/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:05.149524 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb756f28-5eec-4698-be87-af56b4b1cf54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88150ecec31e4f4d9a6767523a8571f2, user_id=ab4a7429267a4b70a4a30c8647fd5437, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:05.152505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb756f28-5eec-4698-be87-af56b4b1cf54 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:05.200482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 899/1777] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:05 2022] PUT /identity/v3/projects/88150ecec31e4f4d9a6767523a8571f2/users/ab4a7429267a4b70a4a30c8647fd5437/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:05.201643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb756f28-5eec-4698-be87-af56b4b1cf54 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:05.205870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cb756f28-5eec-4698-be87-af56b4b1cf54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:05.250095 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:05.314301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 879/1778] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:05.317360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c685-0005-44b4-a48b-ece336077b85 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:05.321176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c685-0005-44b4-a48b-ece336077b85 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:05.322758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c685-0005-44b4-a48b-ece336077b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:05.388893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb756f28-5eec-4698-be87-af56b4b1cf54 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:05.412877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:05.415453 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:05.416226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:05.416800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1458c685-0005-44b4-a48b-ece336077b85 None None] MFA Rules not processed for user `ab4a7429267a4b70a4a30c8647fd5437`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:05.417124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] PATH_INFO: `/v3/users/ecd552da44894184adfdbb41831d8c1d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:05.418268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1458c685-0005-44b4-a48b-ece336077b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:05.419137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd552da44894184adfdbb41831d8c1d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:05.422670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:05.503914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:05.562455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd552da44894184adfdbb41831d8c1d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:05.566255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd552da44894184adfdbb41831d8c1d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:05.569460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd552da44894184adfdbb41831d8c1d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:05.613025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd552da44894184adfdbb41831d8c1d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:05.630173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd552da44894184adfdbb41831d8c1d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:05.681169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 900/1779] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:05 2022] DELETE /identity/v3/users/ecd552da44894184adfdbb41831d8c1d => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:05.681853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:05.684194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:05.693147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 880/1780] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:05.699578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a3790-bf2f-4a85-a48f-a5903fa5b5de None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:05.699862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a3790-bf2f-4a85-a48f-a5903fa5b5de None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:05.700140 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a3790-bf2f-4a85-a48f-a5903fa5b5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:05.771252 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-bc9a3790-bf2f-4a85-a48f-a5903fa5b5de None None] MFA Rules not processed for user `5e18461c8cc4497eae7a83d9ad1448f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:05.771976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bc9a3790-bf2f-4a85-a48f-a5903fa5b5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:05.871201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd546f3f-d7f4-469b-bade-e931cba26937 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:05.886344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f200a08-cac0-45ee-9f37-0c64b4c34afa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:05.887040 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f200a08-cac0-45ee-9f37-0c64b4c34afa service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:05.887586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f200a08-cac0-45ee-9f37-0c64b4c34afa service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:05.887780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f200a08-cac0-45ee-9f37-0c64b4c34afa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:05.896149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2f200a08-cac0-45ee-9f37-0c64b4c34afa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:05.961854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 881/1781] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Aug 24 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:05.962899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a3790-bf2f-4a85-a48f-a5903fa5b5de None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:05.964246 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bc9a3790-bf2f-4a85-a48f-a5903fa5b5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:06.111602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f200a08-cac0-45ee-9f37-0c64b4c34afa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.116892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f200a08-cac0-45ee-9f37-0c64b4c34afa service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.155942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 901/1782] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:06.158039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce108e9-05d6-4218-befe-af969abc2390 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.160144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce108e9-05d6-4218-befe-af969abc2390 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.161214 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce108e9-05d6-4218-befe-af969abc2390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.270775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6ce108e9-05d6-4218-befe-af969abc2390 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:06.275675 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6ce108e9-05d6-4218-befe-af969abc2390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:06.313132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a3790-bf2f-4a85-a48f-a5903fa5b5de None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.325366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 902/1783] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:06.325366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce108e9-05d6-4218-befe-af969abc2390 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.355399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6ce108e9-05d6-4218-befe-af969abc2390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:06.357334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.362750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.363048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.363325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] PATH_INFO: `/v3/projects/124c68917e2344ec8b73105f70adc44e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.365404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=124c68917e2344ec8b73105f70adc44e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.367484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.401688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce108e9-05d6-4218-befe-af969abc2390 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.432759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318b34e-5506-4d7d-9f40-178b2d2ee514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.433789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318b34e-5506-4d7d-9f40-178b2d2ee514 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.434266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318b34e-5506-4d7d-9f40-178b2d2ee514 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.434731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318b34e-5506-4d7d-9f40-178b2d2ee514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.538033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:06.587042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:06.589264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '124c68917e2344ec8b73105f70adc44e', 'request_id': 'req-eef35311-41ca-4b53-8af4-6230b969dd34'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:06.614177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '124c68917e2344ec8b73105f70adc44e', 'request_id': 'req-eef35311-41ca-4b53-8af4-6230b969dd34'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:06.626300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '124c68917e2344ec8b73105f70adc44e', 'request_id': 'req-eef35311-41ca-4b53-8af4-6230b969dd34'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:06.627959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318b34e-5506-4d7d-9f40-178b2d2ee514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.631937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318b34e-5506-4d7d-9f40-178b2d2ee514 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.652342 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] The token cache is being invalidate because project 124c68917e2344ec8b73105f70adc44e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:06.652669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:06.654246 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:06.657494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 882/1784] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:05 2022] DELETE /identity/v3/projects/124c68917e2344ec8b73105f70adc44e => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:06.661252 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.663253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 903/1785] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:06.664929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:06.682257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318b34e-5506-4d7d-9f40-178b2d2ee514 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.687817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d318b34e-5506-4d7d-9f40-178b2d2ee514 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:06.749290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318b34e-5506-4d7d-9f40-178b2d2ee514 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.894319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef35311-41ca-4b53-8af4-6230b969dd34 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.910596 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d8786-d73b-4c83-a45b-36565a611c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.915747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d8786-d73b-4c83-a45b-36565a611c5a service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.916118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d8786-d73b-4c83-a45b-36565a611c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.916959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d8786-d73b-4c83-a45b-36565a611c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.934846 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3b7d8786-d73b-4c83-a45b-36565a611c5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:06.959622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d8786-d73b-4c83-a45b-36565a611c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.961363 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d8786-d73b-4c83-a45b-36565a611c5a service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:07.010623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 883/1786] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:07.010623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d8786-d73b-4c83-a45b-36565a611c5a service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:07.011898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb92b1-88cb-4bdf-9cdb-255c8e8e7803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:07.011898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb92b1-88cb-4bdf-9cdb-255c8e8e7803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.011898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb92b1-88cb-4bdf-9cdb-255c8e8e7803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.011898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb92b1-88cb-4bdf-9cdb-255c8e8e7803 admin admin] PATH_INFO: `/v3/users/a7eeccf4d26b4030a72695e8f4b5f629` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.013129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb92b1-88cb-4bdf-9cdb-255c8e8e7803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7eeccf4d26b4030a72695e8f4b5f629)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:07.013129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb92b1-88cb-4bdf-9cdb-255c8e8e7803 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:07.020480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d8786-d73b-4c83-a45b-36565a611c5a service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:07.036862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc8249-1eae-48bf-81b7-456fcf357fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:07.041778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc8249-1eae-48bf-81b7-456fcf357fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.042064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc8249-1eae-48bf-81b7-456fcf357fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.042336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc8249-1eae-48bf-81b7-456fcf357fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.053265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eecc8249-1eae-48bf-81b7-456fcf357fa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:07.122862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc8249-1eae-48bf-81b7-456fcf357fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:07.126636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc8249-1eae-48bf-81b7-456fcf357fa6 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:07.157033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 884/1787] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:07.161358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996ac78-404f-41fc-acb8-e776550b2560 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.161682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996ac78-404f-41fc-acb8-e776550b2560 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.162018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996ac78-404f-41fc-acb8-e776550b2560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.211960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f996ac78-404f-41fc-acb8-e776550b2560 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:07.216278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f996ac78-404f-41fc-acb8-e776550b2560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:07.238723 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 885/1788] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:07.240560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996ac78-404f-41fc-acb8-e776550b2560 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:07.247086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f996ac78-404f-41fc-acb8-e776550b2560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:07.266341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0deb92b1-88cb-4bdf-9cdb-255c8e8e7803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7eeccf4d26b4030a72695e8f4b5f629'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.272234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0deb92b1-88cb-4bdf-9cdb-255c8e8e7803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7eeccf4d26b4030a72695e8f4b5f629'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.275812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0deb92b1-88cb-4bdf-9cdb-255c8e8e7803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7eeccf4d26b4030a72695e8f4b5f629'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.334826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0deb92b1-88cb-4bdf-9cdb-255c8e8e7803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7eeccf4d26b4030a72695e8f4b5f629'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.380997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0deb92b1-88cb-4bdf-9cdb-255c8e8e7803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7eeccf4d26b4030a72695e8f4b5f629'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.430601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 904/1789] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:06 2022] DELETE /identity/v3/users/a7eeccf4d26b4030a72695e8f4b5f629 => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:07.432900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e01c5-f59b-4996-a809-7b9f1045ba82 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.433599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e01c5-f59b-4996-a809-7b9f1045ba82 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.434028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e01c5-f59b-4996-a809-7b9f1045ba82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.489277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996ac78-404f-41fc-acb8-e776550b2560 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:07.496838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-bc4e01c5-f59b-4996-a809-7b9f1045ba82 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:07.498328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bc4e01c5-f59b-4996-a809-7b9f1045ba82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:07.631895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70864e8-3bd0-43f4-bd38-8e16f72badb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:07.632947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70864e8-3bd0-43f4-bd38-8e16f72badb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.634916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70864e8-3bd0-43f4-bd38-8e16f72badb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.635261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70864e8-3bd0-43f4-bd38-8e16f72badb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.636984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:07.639602 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d70864e8-3bd0-43f4-bd38-8e16f72badb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:07.664744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 905/1790] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:07.673419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebedda-3d01-43dd-b054-359dc0a11f19 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.674195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebedda-3d01-43dd-b054-359dc0a11f19 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.676055 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebedda-3d01-43dd-b054-359dc0a11f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.687214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:07.758809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6aebedda-3d01-43dd-b054-359dc0a11f19 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:07.761387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6aebedda-3d01-43dd-b054-359dc0a11f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:07.782206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 906/1791] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:07.790615 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebedda-3d01-43dd-b054-359dc0a11f19 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:07.813081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebedda-3d01-43dd-b054-359dc0a11f19 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:07.836216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400bd3a-6ad3-4f5c-b86c-d0b579352f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:07.839751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400bd3a-6ad3-4f5c-b86c-d0b579352f7d admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.839751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400bd3a-6ad3-4f5c-b86c-d0b579352f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.839977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400bd3a-6ad3-4f5c-b86c-d0b579352f7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.840613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400bd3a-6ad3-4f5c-b86c-d0b579352f7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:07.841009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400bd3a-6ad3-4f5c-b86c-d0b579352f7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:07.842584 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400bd3a-6ad3-4f5c-b86c-d0b579352f7d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:07.858185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70864e8-3bd0-43f4-bd38-8e16f72badb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:07.859652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70864e8-3bd0-43f4-bd38-8e16f72badb5 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:07.874419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 907/1792] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:07.876163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 886/1793] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:07.882897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400bd3a-6ad3-4f5c-b86c-d0b579352f7d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:07.895345 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400bd3a-6ad3-4f5c-b86c-d0b579352f7d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:07.902788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e4637-8736-497b-8bb5-6f81307f99bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:07.903694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e4637-8736-497b-8bb5-6f81307f99bd admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.904160 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e4637-8736-497b-8bb5-6f81307f99bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.904626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e4637-8736-497b-8bb5-6f81307f99bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.906881 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e4637-8736-497b-8bb5-6f81307f99bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:07.915886 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e4637-8736-497b-8bb5-6f81307f99bd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:07.982361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 908/1794] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:07 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:07.983580 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70864e8-3bd0-43f4-bd38-8e16f72badb5 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:08.006339 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70864e8-3bd0-43f4-bd38-8e16f72badb5 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:08.029337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e4637-8736-497b-8bb5-6f81307f99bd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:08.031078 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-501e4637-8736-497b-8bb5-6f81307f99bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:08.184331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e4637-8736-497b-8bb5-6f81307f99bd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:08.205900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f1008-7775-4218-a046-32a905176d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:08.206869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f1008-7775-4218-a046-32a905176d29 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:08.207340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f1008-7775-4218-a046-32a905176d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:08.207782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f1008-7775-4218-a046-32a905176d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:08.246829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa76ec-7087-4b9e-8c43-ac12b11fdbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:08.247837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa76ec-7087-4b9e-8c43-ac12b11fdbda admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:08.248268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa76ec-7087-4b9e-8c43-ac12b11fdbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:08.248712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa76ec-7087-4b9e-8c43-ac12b11fdbda admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:08.249553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa76ec-7087-4b9e-8c43-ac12b11fdbda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:08.264918 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa76ec-7087-4b9e-8c43-ac12b11fdbda admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:08.344041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 887/1795] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:07 2022] POST /identity/v3/users => generated 292 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:08.349235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa76ec-7087-4b9e-8c43-ac12b11fdbda admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:08.370689 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa76ec-7087-4b9e-8c43-ac12b11fdbda admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:08.392287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920f738-8efa-45ba-9f81-0270be191bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:08.393859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920f738-8efa-45ba-9f81-0270be191bdf admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:08.394716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920f738-8efa-45ba-9f81-0270be191bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:08.397503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920f738-8efa-45ba-9f81-0270be191bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:08.398544 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920f738-8efa-45ba-9f81-0270be191bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:08.407127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920f738-8efa-45ba-9f81-0270be191bdf admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:08.458759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f1008-7775-4218-a046-32a905176d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:08.461017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f1008-7775-4218-a046-32a905176d29 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:08.469228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 888/1796] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:08.472013 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920f738-8efa-45ba-9f81-0270be191bdf admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:08.490631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 909/1797] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:08.506657 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920f738-8efa-45ba-9f81-0270be191bdf admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:08.517399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577f91f-79ca-4d0a-92df-cfff305385ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:08.518872 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577f91f-79ca-4d0a-92df-cfff305385ee admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:08.519159 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577f91f-79ca-4d0a-92df-cfff305385ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:08.519726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577f91f-79ca-4d0a-92df-cfff305385ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:08.524998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577f91f-79ca-4d0a-92df-cfff305385ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:08.526841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577f91f-79ca-4d0a-92df-cfff305385ee admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:08.577299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 889/1798] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:08.586359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f1008-7775-4218-a046-32a905176d29 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:08.603955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f1008-7775-4218-a046-32a905176d29 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:08.608945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b59a13-d081-432f-a603-9b5631b81e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:08.610052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b59a13-d081-432f-a603-9b5631b81e9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:08.610052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b59a13-d081-432f-a603-9b5631b81e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:08.610436 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b59a13-d081-432f-a603-9b5631b81e9a admin admin] PATH_INFO: `/v3/projects/d53fb79a0070496e92fe23e1c0da1f7d/users/7cb495348fb14851828127ccea35ea22/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:08.690944 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b59a13-d081-432f-a603-9b5631b81e9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d53fb79a0070496e92fe23e1c0da1f7d, user_id=7cb495348fb14851828127ccea35ea22, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:08.709641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b59a13-d081-432f-a603-9b5631b81e9a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:08.758558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 910/1799] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:08 2022] PUT /identity/v3/projects/d53fb79a0070496e92fe23e1c0da1f7d/users/7cb495348fb14851828127ccea35ea22/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:08.769855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb20ba5-9316-423c-82de-034fbb34a23c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:08.770424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb20ba5-9316-423c-82de-034fbb34a23c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:08.770972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb20ba5-9316-423c-82de-034fbb34a23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:08.848122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5eb20ba5-9316-423c-82de-034fbb34a23c None None] MFA Rules not processed for user `7cb495348fb14851828127ccea35ea22`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:08.850526 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5eb20ba5-9316-423c-82de-034fbb34a23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:09.042615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 890/1800] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:09.047212 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b59a13-d081-432f-a603-9b5631b81e9a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.146037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:09.169308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b59a13-d081-432f-a603-9b5631b81e9a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.175731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58228307-c825-4ae0-ba56-e4e0433fc963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.176602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58228307-c825-4ae0-ba56-e4e0433fc963 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.177254 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58228307-c825-4ae0-ba56-e4e0433fc963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.177837 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58228307-c825-4ae0-ba56-e4e0433fc963 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.178685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58228307-c825-4ae0-ba56-e4e0433fc963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.181030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58228307-c825-4ae0-ba56-e4e0433fc963 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.218854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 911/1801] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:09 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:09.223210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb20ba5-9316-423c-82de-034fbb34a23c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.241549 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb20ba5-9316-423c-82de-034fbb34a23c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.278396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:09.424467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a001e-0bad-4196-9b5e-93ba570af0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.425699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a001e-0bad-4196-9b5e-93ba570af0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.426446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a001e-0bad-4196-9b5e-93ba570af0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.426996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a001e-0bad-4196-9b5e-93ba570af0f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.428269 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a001e-0bad-4196-9b5e-93ba570af0f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.434263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a001e-0bad-4196-9b5e-93ba570af0f1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.502206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 891/1802] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:09 2022] POST /identity/v3/users => generated 298 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:09.508911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58228307-c825-4ae0-ba56-e4e0433fc963 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.531972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a001e-0bad-4196-9b5e-93ba570af0f1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.560601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-362a001e-0bad-4196-9b5e-93ba570af0f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:09.568974 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58228307-c825-4ae0-ba56-e4e0433fc963 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.618515 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff57684-79a2-49e9-92ee-f5f7774b4996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.620765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff57684-79a2-49e9-92ee-f5f7774b4996 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.622774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff57684-79a2-49e9-92ee-f5f7774b4996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.623282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff57684-79a2-49e9-92ee-f5f7774b4996 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.624436 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff57684-79a2-49e9-92ee-f5f7774b4996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.635916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff57684-79a2-49e9-92ee-f5f7774b4996 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.652969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a001e-0bad-4196-9b5e-93ba570af0f1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.683915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.684935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.685378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.685813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] PATH_INFO: `/v3/projects/c65f948b7eb24ada8decc4b490a2cba0` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.688523 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] RBAC: Authorizing `identity:delete_project(project_id=c65f948b7eb24ada8decc4b490a2cba0)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.693365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.696273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 912/1803] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:09.710653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff57684-79a2-49e9-92ee-f5f7774b4996 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.794348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff57684-79a2-49e9-92ee-f5f7774b4996 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.803724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26054f3b-177f-4707-86c3-b7ac814dfeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.805178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26054f3b-177f-4707-86c3-b7ac814dfeae admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.806050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26054f3b-177f-4707-86c3-b7ac814dfeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.806793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26054f3b-177f-4707-86c3-b7ac814dfeae admin admin] PATH_INFO: `/v3/projects/aa5615029a42483d88e9298947d37c41/users/124ebd7bae2e4c10ac572c3d898dc555/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.840568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26054f3b-177f-4707-86c3-b7ac814dfeae admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5615029a42483d88e9298947d37c41, user_id=124ebd7bae2e4c10ac572c3d898dc555, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.848973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26054f3b-177f-4707-86c3-b7ac814dfeae admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.914355 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 913/1804] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:09 2022] PUT /identity/v3/projects/aa5615029a42483d88e9298947d37c41/users/124ebd7bae2e4c10ac572c3d898dc555/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:09.917113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26054f3b-177f-4707-86c3-b7ac814dfeae admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:10.100521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c65f948b7eb24ada8decc4b490a2cba0', 'request_id': 'req-8cfb88d2-2a44-4198-9114-175170cd24da'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:10.123653 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c65f948b7eb24ada8decc4b490a2cba0', 'request_id': 'req-8cfb88d2-2a44-4198-9114-175170cd24da'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:10.199819 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c65f948b7eb24ada8decc4b490a2cba0', 'request_id': 'req-8cfb88d2-2a44-4198-9114-175170cd24da'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:10.265264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] The token cache is being invalidate because project c65f948b7eb24ada8decc4b490a2cba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:10.265589 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:10.278850 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:10.284968 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 892/1805] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:09 2022] DELETE /identity/v3/projects/c65f948b7eb24ada8decc4b490a2cba0 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:10.290367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:10.306548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:10.354253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26054f3b-177f-4707-86c3-b7ac814dfeae admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:10.584419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824e744-7664-4873-b091-610e3552e2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:10.585102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824e744-7664-4873-b091-610e3552e2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.585369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824e744-7664-4873-b091-610e3552e2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.585641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824e744-7664-4873-b091-610e3552e2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.586293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824e744-7664-4873-b091-610e3552e2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:10.588376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824e744-7664-4873-b091-610e3552e2e9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:10.620919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb88d2-2a44-4198-9114-175170cd24da admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:10.623397 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 914/1806] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 708 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:10.624295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824e744-7664-4873-b091-610e3552e2e9 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:10.632982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e824e744-7664-4873-b091-610e3552e2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:10.650153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5d5a1-641d-4292-b7b1-7f0c26faa5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:10.650889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d5a1-641d-4292-b7b1-7f0c26faa5ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.651169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d5a1-641d-4292-b7b1-7f0c26faa5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.651443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d5a1-641d-4292-b7b1-7f0c26faa5ff admin admin] PATH_INFO: `/v3/users/8bf7713683bb4acba27510a16568eae0` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.658208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5d5a1-641d-4292-b7b1-7f0c26faa5ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf7713683bb4acba27510a16568eae0)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:10.663727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5d5a1-641d-4292-b7b1-7f0c26faa5ff admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:10.687893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824e744-7664-4873-b091-610e3552e2e9 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:10.707312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65e66b-7295-42ed-a4d5-22eaddf2ee53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:10.708011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65e66b-7295-42ed-a4d5-22eaddf2ee53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.708297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65e66b-7295-42ed-a4d5-22eaddf2ee53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.708620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65e66b-7295-42ed-a4d5-22eaddf2ee53 admin admin] PATH_INFO: `/v3/projects/aa5615029a42483d88e9298947d37c41/users/124ebd7bae2e4c10ac572c3d898dc555/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.721812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65e66b-7295-42ed-a4d5-22eaddf2ee53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5615029a42483d88e9298947d37c41, user_id=124ebd7bae2e4c10ac572c3d898dc555, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:10.726972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65e66b-7295-42ed-a4d5-22eaddf2ee53 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:10.753617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 915/1807] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:10 2022] PUT /identity/v3/projects/aa5615029a42483d88e9298947d37c41/users/124ebd7bae2e4c10ac572c3d898dc555/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:10.764235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221f24c-2050-4efa-9017-eec4ee02305b None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.764521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221f24c-2050-4efa-9017-eec4ee02305b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.772250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221f24c-2050-4efa-9017-eec4ee02305b None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.777075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 916/1808] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:33:10 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:33:10.778476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8f16a-dea9-431a-ae6a-1eb4b8d30e5f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.778963 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8f16a-dea9-431a-ae6a-1eb4b8d30e5f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.779086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8f16a-dea9-431a-ae6a-1eb4b8d30e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.800876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:10.884062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-26e8f16a-dea9-431a-ae6a-1eb4b8d30e5f None None] MFA Rules not processed for user `124ebd7bae2e4c10ac572c3d898dc555`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:10.885747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-26e8f16a-dea9-431a-ae6a-1eb4b8d30e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:10.887947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0bf5d5a1-641d-4292-b7b1-7f0c26faa5ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf7713683bb4acba27510a16568eae0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:10.890685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0bf5d5a1-641d-4292-b7b1-7f0c26faa5ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf7713683bb4acba27510a16568eae0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:10.904405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0bf5d5a1-641d-4292-b7b1-7f0c26faa5ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf7713683bb4acba27510a16568eae0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:10.905107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:10.954755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0bf5d5a1-641d-4292-b7b1-7f0c26faa5ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf7713683bb4acba27510a16568eae0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:10.981923 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0bf5d5a1-641d-4292-b7b1-7f0c26faa5ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf7713683bb4acba27510a16568eae0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:11.027969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 893/1809] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:10 2022] DELETE /identity/v3/users/8bf7713683bb4acba27510a16568eae0 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:11.029693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a3019-78cd-47df-9deb-9bad402ac9d7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:11.030783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a3019-78cd-47df-9deb-9bad402ac9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:11.032168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a3019-78cd-47df-9deb-9bad402ac9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:11.092889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5e8a3019-78cd-47df-9deb-9bad402ac9d7 None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:11.093201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 917/1810] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:11.094556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5e8a3019-78cd-47df-9deb-9bad402ac9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:11.100271 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8f16a-dea9-431a-ae6a-1eb4b8d30e5f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:11.102563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-26e8f16a-dea9-431a-ae6a-1eb4b8d30e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:11.239227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8f16a-dea9-431a-ae6a-1eb4b8d30e5f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:11.328115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 894/1811] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:11.407051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f4471-4b1c-4def-ab9f-990332147363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:11.410033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f4471-4b1c-4def-ab9f-990332147363 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:11.410433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f4471-4b1c-4def-ab9f-990332147363 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:11.410773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f4471-4b1c-4def-ab9f-990332147363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:11.416674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-511f4471-4b1c-4def-ab9f-990332147363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:11.655024 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f4471-4b1c-4def-ab9f-990332147363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:11.665186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f4471-4b1c-4def-ab9f-990332147363 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:11.701990 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a3019-78cd-47df-9deb-9bad402ac9d7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:11.703684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5e8a3019-78cd-47df-9deb-9bad402ac9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:11.720426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 918/1812] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:11.734049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a3019-78cd-47df-9deb-9bad402ac9d7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:11.748533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f4471-4b1c-4def-ab9f-990332147363 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:11.750576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-511f4471-4b1c-4def-ab9f-990332147363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:11.758722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a35bd4-91dc-4328-a368-2448673c46b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:11.760094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a35bd4-91dc-4328-a368-2448673c46b7 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:11.760597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a35bd4-91dc-4328-a368-2448673c46b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:11.761320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a35bd4-91dc-4328-a368-2448673c46b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:11.798598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a35bd4-91dc-4328-a368-2448673c46b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:11.800063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:11.809718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a35bd4-91dc-4328-a368-2448673c46b7 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:11.904355 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 895/1813] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:11.992669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f4471-4b1c-4def-ab9f-990332147363 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:12.008165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:12.009816 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:12.010223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:12.013379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] PATH_INFO: `/v3/projects/0f046f87485a425ea099faccf8e3e1c5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:12.023770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f046f87485a425ea099faccf8e3e1c5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:12.026094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:12.166440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f046f87485a425ea099faccf8e3e1c5', 'request_id': 'req-641f357b-ea0c-4487-a495-677204358356'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.186551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f046f87485a425ea099faccf8e3e1c5', 'request_id': 'req-641f357b-ea0c-4487-a495-677204358356'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.195810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f046f87485a425ea099faccf8e3e1c5', 'request_id': 'req-641f357b-ea0c-4487-a495-677204358356'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.225648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] The token cache is being invalidate because project 0f046f87485a425ea099faccf8e3e1c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:12.225962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.228129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.238555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 919/1814] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:11 2022] DELETE /identity/v3/projects/0f046f87485a425ea099faccf8e3e1c5 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:12.242876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a35bd4-91dc-4328-a368-2448673c46b7 service nova] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:12.261168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a35bd4-91dc-4328-a368-2448673c46b7 service nova] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:12.420834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a605fb-3315-4a4f-9340-9e702fedd6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:12.421606 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a605fb-3315-4a4f-9340-9e702fedd6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:12.422136 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a605fb-3315-4a4f-9340-9e702fedd6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:12.422422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a605fb-3315-4a4f-9340-9e702fedd6c9 admin admin] PATH_INFO: `/v3/users/086d2324599348e5a1ece04315109289` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:12.430200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a605fb-3315-4a4f-9340-9e702fedd6c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=086d2324599348e5a1ece04315109289)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:12.432511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a605fb-3315-4a4f-9340-9e702fedd6c9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:12.473944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:12.558053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e8a605fb-3315-4a4f-9340-9e702fedd6c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '086d2324599348e5a1ece04315109289'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.564889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e8a605fb-3315-4a4f-9340-9e702fedd6c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '086d2324599348e5a1ece04315109289'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.566227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e8a605fb-3315-4a4f-9340-9e702fedd6c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '086d2324599348e5a1ece04315109289'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.602917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e8a605fb-3315-4a4f-9340-9e702fedd6c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '086d2324599348e5a1ece04315109289'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.618845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e8a605fb-3315-4a4f-9340-9e702fedd6c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '086d2324599348e5a1ece04315109289'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.646576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 896/1815] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:12 2022] DELETE /identity/v3/users/086d2324599348e5a1ece04315109289 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:12.845568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:12.847158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:12.870432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f357b-ea0c-4487-a495-677204358356 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:12.994859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e6749-37ad-4f08-b29b-445f913c7268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:12.995676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6749-37ad-4f08-b29b-445f913c7268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:12.996004 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6749-37ad-4f08-b29b-445f913c7268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:12.996603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6749-37ad-4f08-b29b-445f913c7268 admin admin] PATH_INFO: `/v3/projects/a3011f05ab064ce998a1a86c9a3ad884` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:12.999257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e6749-37ad-4f08-b29b-445f913c7268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3011f05ab064ce998a1a86c9a3ad884)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:13.001118 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e6749-37ad-4f08-b29b-445f913c7268 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:13.145848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ef5e6749-37ad-4f08-b29b-445f913c7268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3011f05ab064ce998a1a86c9a3ad884', 'request_id': 'req-ef5e6749-37ad-4f08-b29b-445f913c7268'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.150101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:13.163641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ef5e6749-37ad-4f08-b29b-445f913c7268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3011f05ab064ce998a1a86c9a3ad884', 'request_id': 'req-ef5e6749-37ad-4f08-b29b-445f913c7268'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.177351 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ef5e6749-37ad-4f08-b29b-445f913c7268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3011f05ab064ce998a1a86c9a3ad884', 'request_id': 'req-ef5e6749-37ad-4f08-b29b-445f913c7268'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.201636 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ef5e6749-37ad-4f08-b29b-445f913c7268 admin admin] The token cache is being invalidate because project a3011f05ab064ce998a1a86c9a3ad884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:13.202497 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ef5e6749-37ad-4f08-b29b-445f913c7268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.203000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ef5e6749-37ad-4f08-b29b-445f913c7268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.216774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 920/1816] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:12 2022] DELETE /identity/v3/projects/a3011f05ab064ce998a1a86c9a3ad884 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:13.219530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed861ca-d28c-4296-9aa8-2270448efe26 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:13.219821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed861ca-d28c-4296-9aa8-2270448efe26 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:13.220182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed861ca-d28c-4296-9aa8-2270448efe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:13.258133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-aed861ca-d28c-4296-9aa8-2270448efe26 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:13.259795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-aed861ca-d28c-4296-9aa8-2270448efe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:13.370488 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 897/1817] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:13.378459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f59d4-4ba7-4644-bd14-a550a8b71e50 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:13.379065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f59d4-4ba7-4644-bd14-a550a8b71e50 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:13.379461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f59d4-4ba7-4644-bd14-a550a8b71e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:13.428105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-935f59d4-4ba7-4644-bd14-a550a8b71e50 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:13.431476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-935f59d4-4ba7-4644-bd14-a550a8b71e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:13.452600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 921/1818] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:13.477232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed861ca-d28c-4296-9aa8-2270448efe26 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:13.524920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed861ca-d28c-4296-9aa8-2270448efe26 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:13.540760 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b76ac9-4b49-469e-83ea-d268638de243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:13.541623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b76ac9-4b49-469e-83ea-d268638de243 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:13.542041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b76ac9-4b49-469e-83ea-d268638de243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:13.542543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b76ac9-4b49-469e-83ea-d268638de243 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:13.543364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b76ac9-4b49-469e-83ea-d268638de243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:13.543889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b76ac9-4b49-469e-83ea-d268638de243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:13.545777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b76ac9-4b49-469e-83ea-d268638de243 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:13.606977 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 898/1819] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:13.609778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f59d4-4ba7-4644-bd14-a550a8b71e50 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:13.633908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f59d4-4ba7-4644-bd14-a550a8b71e50 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:13.645781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f50dd-a9a2-46db-99c7-426b5e32822c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:13.652891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f50dd-a9a2-46db-99c7-426b5e32822c admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:13.652891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f50dd-a9a2-46db-99c7-426b5e32822c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:13.652891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f50dd-a9a2-46db-99c7-426b5e32822c admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:13.652891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f50dd-a9a2-46db-99c7-426b5e32822c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:13.652891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f50dd-a9a2-46db-99c7-426b5e32822c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:13.760025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 922/1820] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:13 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:13.764023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b76ac9-4b49-469e-83ea-d268638de243 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:13.781625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b76ac9-4b49-469e-83ea-d268638de243 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:14.034376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6f706-57c9-4f39-94bd-881ad78f73da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:14.036859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6f706-57c9-4f39-94bd-881ad78f73da admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:14.037878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6f706-57c9-4f39-94bd-881ad78f73da admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:14.038696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6f706-57c9-4f39-94bd-881ad78f73da admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:14.042067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6f706-57c9-4f39-94bd-881ad78f73da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:14.053099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6f706-57c9-4f39-94bd-881ad78f73da admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:14.139066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 899/1821] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:13 2022] POST /identity/v3/users => generated 284 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:14.140198 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f50dd-a9a2-46db-99c7-426b5e32822c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:14.153548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f50dd-a9a2-46db-99c7-426b5e32822c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:14.170845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f555605-8997-4aa7-a52c-58abbda0d084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:14.171482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f555605-8997-4aa7-a52c-58abbda0d084 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:14.171759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f555605-8997-4aa7-a52c-58abbda0d084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:14.172028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f555605-8997-4aa7-a52c-58abbda0d084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:14.172748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f555605-8997-4aa7-a52c-58abbda0d084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:14.174876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f555605-8997-4aa7-a52c-58abbda0d084 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:14.205826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 923/1822] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:14.219338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6f706-57c9-4f39-94bd-881ad78f73da admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:14.231920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6f706-57c9-4f39-94bd-881ad78f73da admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:14.251033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def22b58-1b1f-47a6-a1b1-f9809880c2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:14.253765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def22b58-1b1f-47a6-a1b1-f9809880c2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:14.257660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def22b58-1b1f-47a6-a1b1-f9809880c2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:14.258802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def22b58-1b1f-47a6-a1b1-f9809880c2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:14.260448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def22b58-1b1f-47a6-a1b1-f9809880c2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:14.264278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def22b58-1b1f-47a6-a1b1-f9809880c2ea admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:14.300109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 900/1823] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:14.304092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f555605-8997-4aa7-a52c-58abbda0d084 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:14.320294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f555605-8997-4aa7-a52c-58abbda0d084 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:14.342309 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb239de-c550-4ef9-9cd7-54dd8aeef7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:14.348028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb239de-c550-4ef9-9cd7-54dd8aeef7d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:14.348314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb239de-c550-4ef9-9cd7-54dd8aeef7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:14.348592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb239de-c550-4ef9-9cd7-54dd8aeef7d2 admin admin] PATH_INFO: `/v3/projects/6ce91e5d02e5408a95ff28e3edcaff44/users/c3f1842e2f914177b04fe569019ae12b/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:14.422395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb239de-c550-4ef9-9cd7-54dd8aeef7d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce91e5d02e5408a95ff28e3edcaff44, user_id=c3f1842e2f914177b04fe569019ae12b, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:14.430812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb239de-c550-4ef9-9cd7-54dd8aeef7d2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:14.484054 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 924/1824] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:14 2022] PUT /identity/v3/projects/6ce91e5d02e5408a95ff28e3edcaff44/users/c3f1842e2f914177b04fe569019ae12b/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:14.484787 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5d394-10f5-483b-8c60-a8b2d2a125e9 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:14.485361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5d394-10f5-483b-8c60-a8b2d2a125e9 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:14.485519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5d394-10f5-483b-8c60-a8b2d2a125e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:14.521651 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:14.575649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3aa5d394-10f5-483b-8c60-a8b2d2a125e9 None None] MFA Rules not processed for user `c3f1842e2f914177b04fe569019ae12b`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:14.577034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3aa5d394-10f5-483b-8c60-a8b2d2a125e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:14.757340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb239de-c550-4ef9-9cd7-54dd8aeef7d2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:14.759486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 901/1825] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:14.923389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb239de-c550-4ef9-9cd7-54dd8aeef7d2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:14.950312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a0695-6341-47ab-be3e-157afbadb007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:14.951235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a0695-6341-47ab-be3e-157afbadb007 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:14.951509 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a0695-6341-47ab-be3e-157afbadb007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:14.951777 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a0695-6341-47ab-be3e-157afbadb007 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:14.952409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a0695-6341-47ab-be3e-157afbadb007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:14.965665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a0695-6341-47ab-be3e-157afbadb007 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.068732 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5d394-10f5-483b-8c60-a8b2d2a125e9 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.082934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 925/1826] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:14 2022] POST /identity/v3/projects => generated 373 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:15.093660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5d394-10f5-483b-8c60-a8b2d2a125e9 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.243467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294cddb-f246-4c5d-8eba-2dca8ababb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.244206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294cddb-f246-4c5d-8eba-2dca8ababb56 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.245728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294cddb-f246-4c5d-8eba-2dca8ababb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.246302 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294cddb-f246-4c5d-8eba-2dca8ababb56 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.247506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294cddb-f246-4c5d-8eba-2dca8ababb56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.254217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294cddb-f246-4c5d-8eba-2dca8ababb56 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.264368 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a0695-6341-47ab-be3e-157afbadb007 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.285335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-000a0695-6341-47ab-be3e-157afbadb007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:15.323441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a0695-6341-47ab-be3e-157afbadb007 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.343919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde6411-9fb8-42a1-9b25-d72464924186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.345013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde6411-9fb8-42a1-9b25-d72464924186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.348724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde6411-9fb8-42a1-9b25-d72464924186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.349219 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde6411-9fb8-42a1-9b25-d72464924186 admin admin] PATH_INFO: `/v3/projects/b8db1e553f8a46c9a27c8900ac394946` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.354161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde6411-9fb8-42a1-9b25-d72464924186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8db1e553f8a46c9a27c8900ac394946)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.358103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 902/1827] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:15 2022] POST /identity/v3/users => generated 293 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:15.359568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294cddb-f246-4c5d-8eba-2dca8ababb56 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.363212 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde6411-9fb8-42a1-9b25-d72464924186 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.374442 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294cddb-f246-4c5d-8eba-2dca8ababb56 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.382006 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d54aab-b2a6-44d6-82c0-53c9574cef8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.383092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d54aab-b2a6-44d6-82c0-53c9574cef8b admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.383642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d54aab-b2a6-44d6-82c0-53c9574cef8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.384120 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d54aab-b2a6-44d6-82c0-53c9574cef8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.385112 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d54aab-b2a6-44d6-82c0-53c9574cef8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.388732 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d54aab-b2a6-44d6-82c0-53c9574cef8b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.413973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 903/1828] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:15.415638 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d54aab-b2a6-44d6-82c0-53c9574cef8b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.435261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d54aab-b2a6-44d6-82c0-53c9574cef8b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.443328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a5f27-a100-4f81-8e04-7d872c5bef99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.445057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a5f27-a100-4f81-8e04-7d872c5bef99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.445541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a5f27-a100-4f81-8e04-7d872c5bef99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.446052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a5f27-a100-4f81-8e04-7d872c5bef99 admin admin] PATH_INFO: `/v3/projects/f1342cfe228a450dabc5438f3bdb81b2/users/b5400eb25f344ceab571e7b343c33298/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.478615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a5f27-a100-4f81-8e04-7d872c5bef99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1342cfe228a450dabc5438f3bdb81b2, user_id=b5400eb25f344ceab571e7b343c33298, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.487639 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a5f27-a100-4f81-8e04-7d872c5bef99 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.519085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-afde6411-9fb8-42a1-9b25-d72464924186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8db1e553f8a46c9a27c8900ac394946', 'request_id': 'req-afde6411-9fb8-42a1-9b25-d72464924186'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.531890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 904/1829] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:15 2022] PUT /identity/v3/projects/f1342cfe228a450dabc5438f3bdb81b2/users/b5400eb25f344ceab571e7b343c33298/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:15.538762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a5f27-a100-4f81-8e04-7d872c5bef99 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.545318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-afde6411-9fb8-42a1-9b25-d72464924186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8db1e553f8a46c9a27c8900ac394946', 'request_id': 'req-afde6411-9fb8-42a1-9b25-d72464924186'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.547638 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:15.563046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-afde6411-9fb8-42a1-9b25-d72464924186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8db1e553f8a46c9a27c8900ac394946', 'request_id': 'req-afde6411-9fb8-42a1-9b25-d72464924186'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.595477 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-afde6411-9fb8-42a1-9b25-d72464924186 admin admin] The token cache is being invalidate because project b8db1e553f8a46c9a27c8900ac394946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:15.598142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-afde6411-9fb8-42a1-9b25-d72464924186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.601392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-afde6411-9fb8-42a1-9b25-d72464924186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.611416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 926/1830] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:15 2022] DELETE /identity/v3/projects/b8db1e553f8a46c9a27c8900ac394946 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:15.613823 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddd7bf-ff67-44b8-9bea-15b6a3f7831f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.615254 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddd7bf-ff67-44b8-9bea-15b6a3f7831f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.615629 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddd7bf-ff67-44b8-9bea-15b6a3f7831f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.678311 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d1ddd7bf-ff67-44b8-9bea-15b6a3f7831f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:15.692999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d1ddd7bf-ff67-44b8-9bea-15b6a3f7831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:15.698429 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:15.728377 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a5f27-a100-4f81-8e04-7d872c5bef99 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.745552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea3a63-a231-4ec9-ad81-e44f47e2fe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.747712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea3a63-a231-4ec9-ad81-e44f47e2fe66 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.750424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea3a63-a231-4ec9-ad81-e44f47e2fe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.752097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea3a63-a231-4ec9-ad81-e44f47e2fe66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.753757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea3a63-a231-4ec9-ad81-e44f47e2fe66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.758680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea3a63-a231-4ec9-ad81-e44f47e2fe66 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.808080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 905/1831] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:15.809227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea3a63-a231-4ec9-ad81-e44f47e2fe66 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.813414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a2ea3a63-a231-4ec9-ad81-e44f47e2fe66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:15.849954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea3a63-a231-4ec9-ad81-e44f47e2fe66 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.869703 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e35c40-618d-4810-a15f-816d910824b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.872310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e35c40-618d-4810-a15f-816d910824b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.874353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e35c40-618d-4810-a15f-816d910824b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.874353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e35c40-618d-4810-a15f-816d910824b8 admin admin] PATH_INFO: `/v3/projects/f1342cfe228a450dabc5438f3bdb81b2/users/b5400eb25f344ceab571e7b343c33298/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.882287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e35c40-618d-4810-a15f-816d910824b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1342cfe228a450dabc5438f3bdb81b2, user_id=b5400eb25f344ceab571e7b343c33298, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.886630 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 927/1832] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:15.893351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e35c40-618d-4810-a15f-816d910824b8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.895181 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162896ef-300a-49a9-b7cd-05a6797b2534 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.895471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162896ef-300a-49a9-b7cd-05a6797b2534 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.895748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162896ef-300a-49a9-b7cd-05a6797b2534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.946661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 906/1833] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:15 2022] PUT /identity/v3/projects/f1342cfe228a450dabc5438f3bdb81b2/users/b5400eb25f344ceab571e7b343c33298/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:15.951420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bcba0-2a34-49e4-8018-94218e4e6e0a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.951708 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bcba0-2a34-49e4-8018-94218e4e6e0a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.951981 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bcba0-2a34-49e4-8018-94218e4e6e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.953815 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-162896ef-300a-49a9-b7cd-05a6797b2534 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:15.953941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-162896ef-300a-49a9-b7cd-05a6797b2534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:16.052279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-cc5bcba0-2a34-49e4-8018-94218e4e6e0a None None] MFA Rules not processed for user `b5400eb25f344ceab571e7b343c33298`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:16.056070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cc5bcba0-2a34-49e4-8018-94218e4e6e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:16.143624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 928/1834] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:16.148119 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162896ef-300a-49a9-b7cd-05a6797b2534 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:16.165778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162896ef-300a-49a9-b7cd-05a6797b2534 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:16.185511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399cc99-f2f0-4b9a-92f5-d64a0263383b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:16.186173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399cc99-f2f0-4b9a-92f5-d64a0263383b admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:16.186455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399cc99-f2f0-4b9a-92f5-d64a0263383b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:16.186731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399cc99-f2f0-4b9a-92f5-d64a0263383b admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:16.187402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399cc99-f2f0-4b9a-92f5-d64a0263383b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:16.187705 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399cc99-f2f0-4b9a-92f5-d64a0263383b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:16.189589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399cc99-f2f0-4b9a-92f5-d64a0263383b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:16.235248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 929/1835] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:16.241682 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399cc99-f2f0-4b9a-92f5-d64a0263383b admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:16.265669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399cc99-f2f0-4b9a-92f5-d64a0263383b admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:16.298143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837ccb0-2984-4f78-896a-9392f4b923f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:16.299554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837ccb0-2984-4f78-896a-9392f4b923f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:16.299841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837ccb0-2984-4f78-896a-9392f4b923f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:16.300364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837ccb0-2984-4f78-896a-9392f4b923f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:16.301141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837ccb0-2984-4f78-896a-9392f4b923f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:16.309064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837ccb0-2984-4f78-896a-9392f4b923f2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:16.336131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 907/1836] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:16.442300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 930/1837] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:16 2022] POST /identity/v3/projects => generated 383 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:16.450832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bcba0-2a34-49e4-8018-94218e4e6e0a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:16.453311 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837ccb0-2984-4f78-896a-9392f4b923f2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:16.455639 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0837ccb0-2984-4f78-896a-9392f4b923f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:16.480290 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bcba0-2a34-49e4-8018-94218e4e6e0a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:16.672540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837ccb0-2984-4f78-896a-9392f4b923f2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:16.681282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093f4f0-3d9b-4136-9881-c35edb620c6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:16.682115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093f4f0-3d9b-4136-9881-c35edb620c6f service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:16.682626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093f4f0-3d9b-4136-9881-c35edb620c6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:16.682942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093f4f0-3d9b-4136-9881-c35edb620c6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:16.696291 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:16.774726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5f63d-abf1-490b-9fb2-40ce0eec9e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:16.776312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5f63d-abf1-490b-9fb2-40ce0eec9e11 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:16.776607 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5f63d-abf1-490b-9fb2-40ce0eec9e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:16.777079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5f63d-abf1-490b-9fb2-40ce0eec9e11 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:16.779401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5f63d-abf1-490b-9fb2-40ce0eec9e11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:16.784705 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5f63d-abf1-490b-9fb2-40ce0eec9e11 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:16.838339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093f4f0-3d9b-4136-9881-c35edb620c6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:16.842822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093f4f0-3d9b-4136-9881-c35edb620c6f service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:16.856230 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 908/1838] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:16 2022] POST /identity/v3/users => generated 292 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:16.861378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5f63d-abf1-490b-9fb2-40ce0eec9e11 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:16.872634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 931/1839] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:16.883549 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5f63d-abf1-490b-9fb2-40ce0eec9e11 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:16.890671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3bec4-d817-4d6b-8e1f-d8af5d596fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:16.891642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3bec4-d817-4d6b-8e1f-d8af5d596fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:16.892555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3bec4-d817-4d6b-8e1f-d8af5d596fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:16.893480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3bec4-d817-4d6b-8e1f-d8af5d596fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:16.894235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3bec4-d817-4d6b-8e1f-d8af5d596fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:16.896372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3bec4-d817-4d6b-8e1f-d8af5d596fdd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:16.915205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 909/1840] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:16.917029 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093f4f0-3d9b-4136-9881-c35edb620c6f service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:16.931244 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093f4f0-3d9b-4136-9881-c35edb620c6f service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:16.937596 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3a3d3-114c-40b5-b379-71c21652e6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:16.938426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3a3d3-114c-40b5-b379-71c21652e6bc admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:16.939141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3a3d3-114c-40b5-b379-71c21652e6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:16.939578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3a3d3-114c-40b5-b379-71c21652e6bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:16.941348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3a3d3-114c-40b5-b379-71c21652e6bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:16.943308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3a3d3-114c-40b5-b379-71c21652e6bc admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:16.958401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 932/1841] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:16.959770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3bec4-d817-4d6b-8e1f-d8af5d596fdd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:16.964634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b40308-4295-4c87-8cd2-62dedcbd46ac None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:16.965017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b40308-4295-4c87-8cd2-62dedcbd46ac None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:16.965488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b40308-4295-4c87-8cd2-62dedcbd46ac None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:16.967184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 933/1842] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:33:17.011814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55f015-dbe8-4b67-9a11-8063782717a4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.012165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55f015-dbe8-4b67-9a11-8063782717a4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.012702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55f015-dbe8-4b67-9a11-8063782717a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.017216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3bec4-d817-4d6b-8e1f-d8af5d596fdd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.041588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bdaa1-43df-4972-9ae2-fac5bed8079b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.042211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bdaa1-43df-4972-9ae2-fac5bed8079b admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.042692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bdaa1-43df-4972-9ae2-fac5bed8079b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.042945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bdaa1-43df-4972-9ae2-fac5bed8079b admin admin] PATH_INFO: `/v3/projects/487246a64bfb4799823bf433b772fd5a/users/9db07d9384bf49dda7c93c5254412388/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.054316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2f55f015-dbe8-4b67-9a11-8063782717a4 None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:17.055605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2f55f015-dbe8-4b67-9a11-8063782717a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:17.098360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bdaa1-43df-4972-9ae2-fac5bed8079b admin admin] RBAC: Authorizing `identity:create_grant(project_id=487246a64bfb4799823bf433b772fd5a, user_id=9db07d9384bf49dda7c93c5254412388, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.101557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bdaa1-43df-4972-9ae2-fac5bed8079b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.122061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 910/1843] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:16 2022] PUT /identity/v3/projects/487246a64bfb4799823bf433b772fd5a/users/9db07d9384bf49dda7c93c5254412388/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:17.134695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976f027-a31e-4a96-ac1a-58e5c388af87 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.135616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976f027-a31e-4a96-ac1a-58e5c388af87 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.138206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976f027-a31e-4a96-ac1a-58e5c388af87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.226022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:17.236150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 934/1844] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:17.258588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f976f027-a31e-4a96-ac1a-58e5c388af87 None None] MFA Rules not processed for user `9db07d9384bf49dda7c93c5254412388`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:17.260230 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f976f027-a31e-4a96-ac1a-58e5c388af87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:17.381795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 911/1845] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:17.386876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55f015-dbe8-4b67-9a11-8063782717a4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.418058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55f015-dbe8-4b67-9a11-8063782717a4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.566468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde8399-b44f-4727-86b0-45c23d580f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.567163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde8399-b44f-4727-86b0-45c23d580f8e admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.567504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde8399-b44f-4727-86b0-45c23d580f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.567839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde8399-b44f-4727-86b0-45c23d580f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.568660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde8399-b44f-4727-86b0-45c23d580f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.570706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde8399-b44f-4727-86b0-45c23d580f8e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.634595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 935/1846] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:17 2022] POST /identity/v3/projects => generated 381 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:17.640436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976f027-a31e-4a96-ac1a-58e5c388af87 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.664270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976f027-a31e-4a96-ac1a-58e5c388af87 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.806866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde8399-b44f-4727-86b0-45c23d580f8e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.810152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8dde8399-b44f-4727-86b0-45c23d580f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:17.853290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:17.913710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb0626-831b-418b-b07b-f97d07a5c4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.914673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb0626-831b-418b-b07b-f97d07a5c4e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.915606 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb0626-831b-418b-b07b-f97d07a5c4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.916331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb0626-831b-418b-b07b-f97d07a5c4e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.919231 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb0626-831b-418b-b07b-f97d07a5c4e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.923145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb0626-831b-418b-b07b-f97d07a5c4e7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:18.032448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 912/1847] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:17 2022] POST /identity/v3/users => generated 297 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:18.042281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb0626-831b-418b-b07b-f97d07a5c4e7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:18.045300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f5cb0626-831b-418b-b07b-f97d07a5c4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:18.078002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde8399-b44f-4727-86b0-45c23d580f8e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:18.089728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc858388-8a7d-4cb1-8cb6-05a6216c6f2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:18.091128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc858388-8a7d-4cb1-8cb6-05a6216c6f2f service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:18.091577 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc858388-8a7d-4cb1-8cb6-05a6216c6f2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:18.092086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc858388-8a7d-4cb1-8cb6-05a6216c6f2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:18.095596 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fc858388-8a7d-4cb1-8cb6-05a6216c6f2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:18.313208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc858388-8a7d-4cb1-8cb6-05a6216c6f2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:18.315404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc858388-8a7d-4cb1-8cb6-05a6216c6f2f service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:18.316256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb0626-831b-418b-b07b-f97d07a5c4e7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:18.331012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ce909-f9f2-431c-a67b-bb6d091da768 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:18.331828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ce909-f9f2-431c-a67b-bb6d091da768 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:18.332226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ce909-f9f2-431c-a67b-bb6d091da768 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:18.332595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ce909-f9f2-431c-a67b-bb6d091da768 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:18.342149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 936/1848] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:18.345052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc858388-8a7d-4cb1-8cb6-05a6216c6f2f service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:18.359563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ce909-f9f2-431c-a67b-bb6d091da768 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:18.364874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ce909-f9f2-431c-a67b-bb6d091da768 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:18.384609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc858388-8a7d-4cb1-8cb6-05a6216c6f2f service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:18.390563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 913/1849] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:18.393913 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ce909-f9f2-431c-a67b-bb6d091da768 service nova] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:18.413453 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb84be-684c-410e-a1d0-9cd9d2df9aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:18.414112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb84be-684c-410e-a1d0-9cd9d2df9aef admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:18.414399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb84be-684c-410e-a1d0-9cd9d2df9aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:18.416893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb84be-684c-410e-a1d0-9cd9d2df9aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:18.420021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb84be-684c-410e-a1d0-9cd9d2df9aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:18.421675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ce909-f9f2-431c-a67b-bb6d091da768 service nova] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:18.432060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb84be-684c-410e-a1d0-9cd9d2df9aef admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:18.435038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f1141-1ea1-4e43-96ba-dd6870e48308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:18.435601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f1141-1ea1-4e43-96ba-dd6870e48308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:18.436259 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f1141-1ea1-4e43-96ba-dd6870e48308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:18.436827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f1141-1ea1-4e43-96ba-dd6870e48308 admin admin] PATH_INFO: `/v3/users/c3f1842e2f914177b04fe569019ae12b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:18.438764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f1141-1ea1-4e43-96ba-dd6870e48308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f1842e2f914177b04fe569019ae12b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:18.441497 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f1141-1ea1-4e43-96ba-dd6870e48308 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:18.459659 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:18.487117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 937/1850] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:18.494188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb84be-684c-410e-a1d0-9cd9d2df9aef admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:18.532820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb84be-684c-410e-a1d0-9cd9d2df9aef admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:18.551568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c4768-62b9-4978-9046-119207e88214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:18.551810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d58f1141-1ea1-4e43-96ba-dd6870e48308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f1842e2f914177b04fe569019ae12b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:18.552858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d58f1141-1ea1-4e43-96ba-dd6870e48308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f1842e2f914177b04fe569019ae12b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:18.553564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d58f1141-1ea1-4e43-96ba-dd6870e48308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f1842e2f914177b04fe569019ae12b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:18.556797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c4768-62b9-4978-9046-119207e88214 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:18.557374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c4768-62b9-4978-9046-119207e88214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:18.557562 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c4768-62b9-4978-9046-119207e88214 admin admin] PATH_INFO: `/v3/projects/740ce760f8d94a8caec7ab19c0d12a4c/users/d880d63c3185437b9d464bb889feb055/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:18.573894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d58f1141-1ea1-4e43-96ba-dd6870e48308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f1842e2f914177b04fe569019ae12b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:18.593646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d58f1141-1ea1-4e43-96ba-dd6870e48308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f1842e2f914177b04fe569019ae12b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:18.609914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c4768-62b9-4978-9046-119207e88214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=740ce760f8d94a8caec7ab19c0d12a4c, user_id=d880d63c3185437b9d464bb889feb055, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:18.617013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c4768-62b9-4978-9046-119207e88214 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:18.625440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 914/1851] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:18 2022] DELETE /identity/v3/users/c3f1842e2f914177b04fe569019ae12b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:18.630330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61012-cc0b-47e4-a333-52c6d94b86e2 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:18.630330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61012-cc0b-47e4-a333-52c6d94b86e2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:18.630851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61012-cc0b-47e4-a333-52c6d94b86e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:18.647255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 938/1852] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:18 2022] PUT /identity/v3/projects/740ce760f8d94a8caec7ab19c0d12a4c/users/d880d63c3185437b9d464bb889feb055/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:18.652236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c4768-62b9-4978-9046-119207e88214 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:18.654314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b93c4768-62b9-4978-9046-119207e88214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:18.656009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-27e61012-cc0b-47e4-a333-52c6d94b86e2 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:18.658738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-27e61012-cc0b-47e4-a333-52c6d94b86e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:18.782297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c4768-62b9-4978-9046-119207e88214 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:18.789728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65a144-bb21-47af-82e5-eaf738406f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:18.791074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65a144-bb21-47af-82e5-eaf738406f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:18.791439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65a144-bb21-47af-82e5-eaf738406f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:18.791853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65a144-bb21-47af-82e5-eaf738406f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:18.792755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65a144-bb21-47af-82e5-eaf738406f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:18.794678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65a144-bb21-47af-82e5-eaf738406f67 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:18.801916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 915/1853] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:18.815109 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 939/1854] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:18.817025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e61012-cc0b-47e4-a333-52c6d94b86e2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:18.822390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65a144-bb21-47af-82e5-eaf738406f67 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:18.828257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0f65a144-bb21-47af-82e5-eaf738406f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:18.832388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e61012-cc0b-47e4-a333-52c6d94b86e2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:18.838952 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adc5e1-ff07-4ccd-a246-3893ae506f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:18.839804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adc5e1-ff07-4ccd-a246-3893ae506f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:18.840110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adc5e1-ff07-4ccd-a246-3893ae506f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:18.840405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adc5e1-ff07-4ccd-a246-3893ae506f45 admin admin] PATH_INFO: `/v3/projects/740ce760f8d94a8caec7ab19c0d12a4c/users/d880d63c3185437b9d464bb889feb055/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:18.842882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62adc5e1-ff07-4ccd-a246-3893ae506f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=740ce760f8d94a8caec7ab19c0d12a4c, user_id=d880d63c3185437b9d464bb889feb055, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:18.847130 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62adc5e1-ff07-4ccd-a246-3893ae506f45 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:18.858145 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:18.874416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 916/1855] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:18 2022] PUT /identity/v3/projects/740ce760f8d94a8caec7ab19c0d12a4c/users/d880d63c3185437b9d464bb889feb055/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:18.879104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f837cb0-7048-4880-8ba4-85e19e589c01 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:18.879395 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f837cb0-7048-4880-8ba4-85e19e589c01 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:18.879671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f837cb0-7048-4880-8ba4-85e19e589c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:18.921648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65a144-bb21-47af-82e5-eaf738406f67 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:18.940581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-0f837cb0-7048-4880-8ba4-85e19e589c01 None None] MFA Rules not processed for user `d880d63c3185437b9d464bb889feb055`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:18.941836 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0f837cb0-7048-4880-8ba4-85e19e589c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:19.013531 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e3948-b1f2-4f54-9192-b8363246e771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:19.014137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e3948-b1f2-4f54-9192-b8363246e771 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:19.014512 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e3948-b1f2-4f54-9192-b8363246e771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:19.014910 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e3948-b1f2-4f54-9192-b8363246e771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:19.080216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 917/1856] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:19.092836 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f837cb0-7048-4880-8ba4-85e19e589c01 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:19.095278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0f837cb0-7048-4880-8ba4-85e19e589c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:19.102337 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:19.123764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e3948-b1f2-4f54-9192-b8363246e771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:19.125379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e3948-b1f2-4f54-9192-b8363246e771 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:19.136987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 940/1857] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:19.185615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f837cb0-7048-4880-8ba4-85e19e589c01 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:19.192249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd4cfe-e84f-4d28-a838-b51e674cb732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:19.193923 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd4cfe-e84f-4d28-a838-b51e674cb732 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:19.195143 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd4cfe-e84f-4d28-a838-b51e674cb732 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:19.195143 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd4cfe-e84f-4d28-a838-b51e674cb732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:19.209239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd4cfe-e84f-4d28-a838-b51e674cb732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:19.211719 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd4cfe-e84f-4d28-a838-b51e674cb732 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:19.230063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 918/1858] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:19.351372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e3948-b1f2-4f54-9192-b8363246e771 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:19.378563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e3948-b1f2-4f54-9192-b8363246e771 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:19.383256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0823a7-4148-425c-8fc5-25ad8c36a2a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:19.383940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0823a7-4148-425c-8fc5-25ad8c36a2a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:19.385575 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0823a7-4148-425c-8fc5-25ad8c36a2a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:19.385863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0823a7-4148-425c-8fc5-25ad8c36a2a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:19.388440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1e0823a7-4148-425c-8fc5-25ad8c36a2a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:19.456029 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0823a7-4148-425c-8fc5-25ad8c36a2a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:19.457558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0823a7-4148-425c-8fc5-25ad8c36a2a8 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:19.467381 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 941/1859] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:20.222226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd4cfe-e84f-4d28-a838-b51e674cb732 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:20.226265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-defd4cfe-e84f-4d28-a838-b51e674cb732 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:20.254697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd4cfe-e84f-4d28-a838-b51e674cb732 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:20.269664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:20.270659 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:20.273334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:20.273793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] PATH_INFO: `/v3/projects/6ce91e5d02e5408a95ff28e3edcaff44` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:20.275559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce91e5d02e5408a95ff28e3edcaff44)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:20.288310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:20.423750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce91e5d02e5408a95ff28e3edcaff44', 'request_id': 'req-8c1bb319-d1f2-4961-8178-781972efd489'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:20.447109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce91e5d02e5408a95ff28e3edcaff44', 'request_id': 'req-8c1bb319-d1f2-4961-8178-781972efd489'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:20.459769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce91e5d02e5408a95ff28e3edcaff44', 'request_id': 'req-8c1bb319-d1f2-4961-8178-781972efd489'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:20.481179 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] The token cache is being invalidate because project 6ce91e5d02e5408a95ff28e3edcaff44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:20.481477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:20.482152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:20.484994 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 919/1860] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:20 2022] DELETE /identity/v3/projects/6ce91e5d02e5408a95ff28e3edcaff44 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:20.489066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0823a7-4148-425c-8fc5-25ad8c36a2a8 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:20.510892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0823a7-4148-425c-8fc5-25ad8c36a2a8 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:20.547239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:20.600007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88211a-40c4-4156-a669-cf1e3179d43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:20.600729 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88211a-40c4-4156-a669-cf1e3179d43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:20.600992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88211a-40c4-4156-a669-cf1e3179d43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:20.601380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88211a-40c4-4156-a669-cf1e3179d43a admin admin] PATH_INFO: `/v3/users/b5400eb25f344ceab571e7b343c33298` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:20.602590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88211a-40c4-4156-a669-cf1e3179d43a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5400eb25f344ceab571e7b343c33298)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:20.604544 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88211a-40c4-4156-a669-cf1e3179d43a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:20.683115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8d88211a-40c4-4156-a669-cf1e3179d43a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5400eb25f344ceab571e7b343c33298'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:20.683929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8d88211a-40c4-4156-a669-cf1e3179d43a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5400eb25f344ceab571e7b343c33298'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:20.684555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8d88211a-40c4-4156-a669-cf1e3179d43a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5400eb25f344ceab571e7b343c33298'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:20.702606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8d88211a-40c4-4156-a669-cf1e3179d43a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5400eb25f344ceab571e7b343c33298'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:20.712878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8d88211a-40c4-4156-a669-cf1e3179d43a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5400eb25f344ceab571e7b343c33298'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:20.732967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 942/1861] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:20 2022] DELETE /identity/v3/users/b5400eb25f344ceab571e7b343c33298 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:21.166430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:21.182303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:21.198012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bb319-d1f2-4961-8178-781972efd489 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:21.279379 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5b276-92f2-443b-bfc9-97e02e8c23dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:21.280037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5b276-92f2-443b-bfc9-97e02e8c23dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:21.280278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5b276-92f2-443b-bfc9-97e02e8c23dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:21.280544 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5b276-92f2-443b-bfc9-97e02e8c23dd admin admin] PATH_INFO: `/v3/projects/f1342cfe228a450dabc5438f3bdb81b2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:21.281848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5b276-92f2-443b-bfc9-97e02e8c23dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1342cfe228a450dabc5438f3bdb81b2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:21.283844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5b276-92f2-443b-bfc9-97e02e8c23dd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:21.326140 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:21.341102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-25b5b276-92f2-443b-bfc9-97e02e8c23dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1342cfe228a450dabc5438f3bdb81b2', 'request_id': 'req-25b5b276-92f2-443b-bfc9-97e02e8c23dd'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:21.349661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-25b5b276-92f2-443b-bfc9-97e02e8c23dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1342cfe228a450dabc5438f3bdb81b2', 'request_id': 'req-25b5b276-92f2-443b-bfc9-97e02e8c23dd'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:21.356419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-25b5b276-92f2-443b-bfc9-97e02e8c23dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1342cfe228a450dabc5438f3bdb81b2', 'request_id': 'req-25b5b276-92f2-443b-bfc9-97e02e8c23dd'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:21.368336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-25b5b276-92f2-443b-bfc9-97e02e8c23dd admin admin] The token cache is being invalidate because project f1342cfe228a450dabc5438f3bdb81b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:21.368678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-25b5b276-92f2-443b-bfc9-97e02e8c23dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:21.369344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-25b5b276-92f2-443b-bfc9-97e02e8c23dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:21.371068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 920/1862] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:21 2022] DELETE /identity/v3/projects/f1342cfe228a450dabc5438f3bdb81b2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:21.376406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e851740-6e6a-4d00-9c42-5968356f76be None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:21.377125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e851740-6e6a-4d00-9c42-5968356f76be None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:21.377681 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e851740-6e6a-4d00-9c42-5968356f76be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:21.399370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7e851740-6e6a-4d00-9c42-5968356f76be None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:21.400595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7e851740-6e6a-4d00-9c42-5968356f76be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:21.472481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 943/1863] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:21.480458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e99959-895a-4ce7-9ebe-eaf10b0cd3a5 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:21.481037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e99959-895a-4ce7-9ebe-eaf10b0cd3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:21.481523 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e99959-895a-4ce7-9ebe-eaf10b0cd3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:21.511568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-85e99959-895a-4ce7-9ebe-eaf10b0cd3a5 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:21.514778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-85e99959-895a-4ce7-9ebe-eaf10b0cd3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:21.526430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 921/1864] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:21.531541 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e851740-6e6a-4d00-9c42-5968356f76be None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:21.546404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e851740-6e6a-4d00-9c42-5968356f76be None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:21.558532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044988db-b65c-487c-9826-720debbdffb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:21.559816 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044988db-b65c-487c-9826-720debbdffb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:21.560094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044988db-b65c-487c-9826-720debbdffb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:21.560513 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044988db-b65c-487c-9826-720debbdffb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:21.561328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044988db-b65c-487c-9826-720debbdffb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:21.561905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044988db-b65c-487c-9826-720debbdffb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:21.565778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044988db-b65c-487c-9826-720debbdffb8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:21.583398 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 944/1865] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:33:21.588132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e99959-895a-4ce7-9ebe-eaf10b0cd3a5 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:21.597813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e99959-895a-4ce7-9ebe-eaf10b0cd3a5 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:21.602972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f253b-709b-4a21-88df-c9ca30e65577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:21.603577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f253b-709b-4a21-88df-c9ca30e65577 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:21.603849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f253b-709b-4a21-88df-c9ca30e65577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:21.604116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f253b-709b-4a21-88df-c9ca30e65577 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:21.604835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f253b-709b-4a21-88df-c9ca30e65577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:21.606820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f253b-709b-4a21-88df-c9ca30e65577 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:21.641954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 922/1866] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:21 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:21.645321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044988db-b65c-487c-9826-720debbdffb8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:21.658973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044988db-b65c-487c-9826-720debbdffb8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:21.700505 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:21.739085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9921e-b6bf-46af-8c0a-4252b267f3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:21.739713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9921e-b6bf-46af-8c0a-4252b267f3f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:21.739992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9921e-b6bf-46af-8c0a-4252b267f3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:21.740262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9921e-b6bf-46af-8c0a-4252b267f3f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:21.740920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9921e-b6bf-46af-8c0a-4252b267f3f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:21.742851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9921e-b6bf-46af-8c0a-4252b267f3f0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:21.769378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 945/1867] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:21 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:21.773503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f253b-709b-4a21-88df-c9ca30e65577 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:21.785933 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f253b-709b-4a21-88df-c9ca30e65577 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:21.790744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ae340-40f3-46c8-92d7-ba82d85c0d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:21.791438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae340-40f3-46c8-92d7-ba82d85c0d29 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:21.791672 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae340-40f3-46c8-92d7-ba82d85c0d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:21.791941 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae340-40f3-46c8-92d7-ba82d85c0d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:21.792586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ae340-40f3-46c8-92d7-ba82d85c0d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:21.794342 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ae340-40f3-46c8-92d7-ba82d85c0d29 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:21.809005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 923/1868] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:21.814283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9921e-b6bf-46af-8c0a-4252b267f3f0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:21.826813 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9921e-b6bf-46af-8c0a-4252b267f3f0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:21.832472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538f27b-c398-49f5-a99e-9f90cbe1d1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:21.833104 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538f27b-c398-49f5-a99e-9f90cbe1d1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:21.833385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538f27b-c398-49f5-a99e-9f90cbe1d1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:21.833642 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538f27b-c398-49f5-a99e-9f90cbe1d1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:21.834282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538f27b-c398-49f5-a99e-9f90cbe1d1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:21.835903 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538f27b-c398-49f5-a99e-9f90cbe1d1ca admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:21.846032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 946/1869] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:21.850263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ae340-40f3-46c8-92d7-ba82d85c0d29 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:21.860409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ae340-40f3-46c8-92d7-ba82d85c0d29 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:21.865419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2bff1-eff4-4240-9d25-d0dfa109afde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:21.866058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2bff1-eff4-4240-9d25-d0dfa109afde admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:21.866331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2bff1-eff4-4240-9d25-d0dfa109afde admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:21.866629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2bff1-eff4-4240-9d25-d0dfa109afde admin admin] PATH_INFO: `/v3/projects/eab3f01e676c44e385a35fa2d2698923/users/f51ca1479899426d90186bb0786a38f2/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:21.882730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2bff1-eff4-4240-9d25-d0dfa109afde admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab3f01e676c44e385a35fa2d2698923, user_id=f51ca1479899426d90186bb0786a38f2, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:21.885622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2bff1-eff4-4240-9d25-d0dfa109afde admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:21.900229 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 924/1870] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:21 2022] PUT /identity/v3/projects/eab3f01e676c44e385a35fa2d2698923/users/f51ca1479899426d90186bb0786a38f2/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:21.916088 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c8a6a-2e36-41da-ab6e-59f1523e83fe None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:21.916444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c8a6a-2e36-41da-ab6e-59f1523e83fe None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:21.916759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c8a6a-2e36-41da-ab6e-59f1523e83fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:21.949844 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f87c8a6a-2e36-41da-ab6e-59f1523e83fe None None] MFA Rules not processed for user `f51ca1479899426d90186bb0786a38f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:21.951105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f87c8a6a-2e36-41da-ab6e-59f1523e83fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:22.023184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 947/1871] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:22.027103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2bff1-eff4-4240-9d25-d0dfa109afde admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:22.106788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2bff1-eff4-4240-9d25-d0dfa109afde admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:22.112753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af593f37-27dc-4783-9f17-2868ab177992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:22.113378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af593f37-27dc-4783-9f17-2868ab177992 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:22.113652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af593f37-27dc-4783-9f17-2868ab177992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:22.113914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af593f37-27dc-4783-9f17-2868ab177992 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:22.114558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af593f37-27dc-4783-9f17-2868ab177992 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:22.116523 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af593f37-27dc-4783-9f17-2868ab177992 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:22.152074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 925/1872] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:22 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:22.153753 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c8a6a-2e36-41da-ab6e-59f1523e83fe None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:22.174470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c8a6a-2e36-41da-ab6e-59f1523e83fe None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:22.259355 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7ed00-6627-4eb6-977c-7d9819866c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:22.260390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7ed00-6627-4eb6-977c-7d9819866c6a admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:22.260690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7ed00-6627-4eb6-977c-7d9819866c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:22.261013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7ed00-6627-4eb6-977c-7d9819866c6a admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:22.261653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7ed00-6627-4eb6-977c-7d9819866c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:22.263679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7ed00-6627-4eb6-977c-7d9819866c6a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:22.303482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af593f37-27dc-4783-9f17-2868ab177992 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:22.306316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-af593f37-27dc-4783-9f17-2868ab177992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:22.317311 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 948/1873] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:22 2022] POST /identity/v3/users => generated 299 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:22.321106 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7ed00-6627-4eb6-977c-7d9819866c6a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:22.332809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7ed00-6627-4eb6-977c-7d9819866c6a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:22.337940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138572aa-4eb9-4359-aa7a-9a01d93253c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:22.338685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138572aa-4eb9-4359-aa7a-9a01d93253c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:22.338995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af593f37-27dc-4783-9f17-2868ab177992 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:22.339141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138572aa-4eb9-4359-aa7a-9a01d93253c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:22.339773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138572aa-4eb9-4359-aa7a-9a01d93253c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:22.340368 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138572aa-4eb9-4359-aa7a-9a01d93253c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:22.342327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138572aa-4eb9-4359-aa7a-9a01d93253c1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:22.346676 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb5f53-55d5-436b-92ea-7eebf85e32e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:22.347559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb5f53-55d5-436b-92ea-7eebf85e32e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:22.347982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb5f53-55d5-436b-92ea-7eebf85e32e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:22.348976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb5f53-55d5-436b-92ea-7eebf85e32e3 admin admin] PATH_INFO: `/v3/users/9db07d9384bf49dda7c93c5254412388` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:22.351545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb5f53-55d5-436b-92ea-7eebf85e32e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9db07d9384bf49dda7c93c5254412388)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:22.354676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 949/1874] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:22.354890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb5f53-55d5-436b-92ea-7eebf85e32e3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:22.358571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138572aa-4eb9-4359-aa7a-9a01d93253c1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:22.372069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138572aa-4eb9-4359-aa7a-9a01d93253c1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:22.380486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee039f-1204-4d78-920e-4b6791261f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:22.383665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee039f-1204-4d78-920e-4b6791261f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:22.383665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee039f-1204-4d78-920e-4b6791261f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:22.383665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee039f-1204-4d78-920e-4b6791261f9a admin admin] PATH_INFO: `/v3/projects/943fc2d5359341e99357c7d27145f754/users/44f86c88f774400abf4307faae26d31d/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:22.402134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:22.406884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee039f-1204-4d78-920e-4b6791261f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=943fc2d5359341e99357c7d27145f754, user_id=44f86c88f774400abf4307faae26d31d, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:22.410620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee039f-1204-4d78-920e-4b6791261f9a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:22.424160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:22.433810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 950/1875] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:22 2022] PUT /identity/v3/projects/943fc2d5359341e99357c7d27145f754/users/44f86c88f774400abf4307faae26d31d/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:22.437565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee039f-1204-4d78-920e-4b6791261f9a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:22.443770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1cfb5f53-55d5-436b-92ea-7eebf85e32e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9db07d9384bf49dda7c93c5254412388'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:22.444783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1cfb5f53-55d5-436b-92ea-7eebf85e32e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9db07d9384bf49dda7c93c5254412388'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:22.446364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1cfb5f53-55d5-436b-92ea-7eebf85e32e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9db07d9384bf49dda7c93c5254412388'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:22.462361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1cfb5f53-55d5-436b-92ea-7eebf85e32e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9db07d9384bf49dda7c93c5254412388'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:22.470996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1cfb5f53-55d5-436b-92ea-7eebf85e32e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9db07d9384bf49dda7c93c5254412388'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:22.487085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 926/1876] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:22 2022] DELETE /identity/v3/users/9db07d9384bf49dda7c93c5254412388 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:22.492294 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12242df-e020-4ef2-bd4a-b3a93329ccd3 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:22.494108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12242df-e020-4ef2-bd4a-b3a93329ccd3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:22.494108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12242df-e020-4ef2-bd4a-b3a93329ccd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:22.520004 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d12242df-e020-4ef2-bd4a-b3a93329ccd3 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:22.521286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d12242df-e020-4ef2-bd4a-b3a93329ccd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:22.538672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee039f-1204-4d78-920e-4b6791261f9a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:22.596038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 927/1877] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:22.605603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb709383-5761-46c8-815c-d493cd75b8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:22.606497 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12242df-e020-4ef2-bd4a-b3a93329ccd3 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:22.606618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb709383-5761-46c8-815c-d493cd75b8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:22.606618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb709383-5761-46c8-815c-d493cd75b8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:22.606931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb709383-5761-46c8-815c-d493cd75b8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:22.607544 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb709383-5761-46c8-815c-d493cd75b8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:22.607737 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d12242df-e020-4ef2-bd4a-b3a93329ccd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:22.609193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb709383-5761-46c8-815c-d493cd75b8e3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:22.624693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 951/1878] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:22.627384 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb709383-5761-46c8-815c-d493cd75b8e3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:22.629630 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-eb709383-5761-46c8-815c-d493cd75b8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:22.645974 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb709383-5761-46c8-815c-d493cd75b8e3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:22.652083 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd441e2-0ea5-40c0-a227-af7ddfe6aa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:22.652753 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd441e2-0ea5-40c0-a227-af7ddfe6aa8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:22.653030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd441e2-0ea5-40c0-a227-af7ddfe6aa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:22.653300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd441e2-0ea5-40c0-a227-af7ddfe6aa8d admin admin] PATH_INFO: `/v3/projects/943fc2d5359341e99357c7d27145f754/users/44f86c88f774400abf4307faae26d31d/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:22.655303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd441e2-0ea5-40c0-a227-af7ddfe6aa8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=943fc2d5359341e99357c7d27145f754, user_id=44f86c88f774400abf4307faae26d31d, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:22.658051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd441e2-0ea5-40c0-a227-af7ddfe6aa8d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:22.679463 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 952/1879] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:22 2022] PUT /identity/v3/projects/943fc2d5359341e99357c7d27145f754/users/44f86c88f774400abf4307faae26d31d/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:22.679767 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12242df-e020-4ef2-bd4a-b3a93329ccd3 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:22.680323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152e35e-9683-4dcf-aac6-29d4c96f70a8 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:22.680577 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152e35e-9683-4dcf-aac6-29d4c96f70a8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:22.680890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152e35e-9683-4dcf-aac6-29d4c96f70a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:22.716941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8152e35e-9683-4dcf-aac6-29d4c96f70a8 None None] MFA Rules not processed for user `44f86c88f774400abf4307faae26d31d`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:22.718521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8152e35e-9683-4dcf-aac6-29d4c96f70a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:22.764491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39718fd-e6f5-4397-a63f-a50894095c48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:22.766945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39718fd-e6f5-4397-a63f-a50894095c48 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:22.767256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39718fd-e6f5-4397-a63f-a50894095c48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:22.767613 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39718fd-e6f5-4397-a63f-a50894095c48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:22.794535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:22.865880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 953/1880] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:22.903429 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152e35e-9683-4dcf-aac6-29d4c96f70a8 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:22.905603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8152e35e-9683-4dcf-aac6-29d4c96f70a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:22.919668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39718fd-e6f5-4397-a63f-a50894095c48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:22.921533 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39718fd-e6f5-4397-a63f-a50894095c48 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:22.934838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 928/1881] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:22.940913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:23.067787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152e35e-9683-4dcf-aac6-29d4c96f70a8 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:23.094236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84455f-34b6-40e4-ab90-f476138f86c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:23.095844 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84455f-34b6-40e4-ab90-f476138f86c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:23.096071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84455f-34b6-40e4-ab90-f476138f86c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:23.096580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84455f-34b6-40e4-ab90-f476138f86c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:23.148075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84455f-34b6-40e4-ab90-f476138f86c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:23.153403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84455f-34b6-40e4-ab90-f476138f86c1 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:23.200099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 954/1882] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:23.264149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39718fd-e6f5-4397-a63f-a50894095c48 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:23.296375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39718fd-e6f5-4397-a63f-a50894095c48 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:23.310665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:23.313499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:23.314339 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:23.315383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] PATH_INFO: `/v3/users/f51ca1479899426d90186bb0786a38f2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:23.321094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f51ca1479899426d90186bb0786a38f2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:23.326927 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:23.474098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f51ca1479899426d90186bb0786a38f2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:23.485601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f51ca1479899426d90186bb0786a38f2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:23.485601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f51ca1479899426d90186bb0786a38f2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:23.537798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f51ca1479899426d90186bb0786a38f2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:23.591984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f51ca1479899426d90186bb0786a38f2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:23.633498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 929/1883] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:23 2022] DELETE /identity/v3/users/f51ca1479899426d90186bb0786a38f2 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:23.636208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035981-24cd-41c9-a00a-0df048ece37d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:23.636758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035981-24cd-41c9-a00a-0df048ece37d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:23.636758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035981-24cd-41c9-a00a-0df048ece37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:23.664418 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d2035981-24cd-41c9-a00a-0df048ece37d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:23.666895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d2035981-24cd-41c9-a00a-0df048ece37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:23.753278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 955/1884] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:23.911347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:23.912978 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:23.927111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cfe77-9e00-462d-bfa3-fec11585ee71 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:23.931960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f7d01-8041-4b6d-a528-26177ad30476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:23.932593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f7d01-8041-4b6d-a528-26177ad30476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:23.932889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f7d01-8041-4b6d-a528-26177ad30476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:23.933164 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f7d01-8041-4b6d-a528-26177ad30476 admin admin] PATH_INFO: `/v3/users/7cb495348fb14851828127ccea35ea22` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:23.934321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f7d01-8041-4b6d-a528-26177ad30476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb495348fb14851828127ccea35ea22)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:23.936253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f7d01-8041-4b6d-a528-26177ad30476 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:23.976835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035981-24cd-41c9-a00a-0df048ece37d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:23.976835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d2035981-24cd-41c9-a00a-0df048ece37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:24.012256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c99f7d01-8041-4b6d-a528-26177ad30476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb495348fb14851828127ccea35ea22'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.013558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c99f7d01-8041-4b6d-a528-26177ad30476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb495348fb14851828127ccea35ea22'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.014348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c99f7d01-8041-4b6d-a528-26177ad30476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb495348fb14851828127ccea35ea22'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.035957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c99f7d01-8041-4b6d-a528-26177ad30476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb495348fb14851828127ccea35ea22'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.046121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c99f7d01-8041-4b6d-a528-26177ad30476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb495348fb14851828127ccea35ea22'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.066296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 930/1885] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:23 2022] DELETE /identity/v3/users/7cb495348fb14851828127ccea35ea22 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:24.072171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0760f-ce71-401d-965d-8e1406e5ae93 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:24.072622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0760f-ce71-401d-965d-8e1406e5ae93 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:24.073296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0760f-ce71-401d-965d-8e1406e5ae93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:24.075879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035981-24cd-41c9-a00a-0df048ece37d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:24.083333 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:24.100786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d3e0760f-ce71-401d-965d-8e1406e5ae93 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:24.102835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d3e0760f-ce71-401d-965d-8e1406e5ae93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:24.155694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb24e5-7c49-4046-8795-0a5681d8f122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:24.156503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb24e5-7c49-4046-8795-0a5681d8f122 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:24.156837 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb24e5-7c49-4046-8795-0a5681d8f122 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:24.157761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb24e5-7c49-4046-8795-0a5681d8f122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:24.159863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f3fb24e5-7c49-4046-8795-0a5681d8f122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:24.185995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:24.186371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 931/1886] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:24.209529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0760f-ce71-401d-965d-8e1406e5ae93 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:24.212306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d3e0760f-ce71-401d-965d-8e1406e5ae93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:24.236017 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0760f-ce71-401d-965d-8e1406e5ae93 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:24.240436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bdc20-d533-4fc5-af2d-081406dbb659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:24.241167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bdc20-d533-4fc5-af2d-081406dbb659 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:24.241510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bdc20-d533-4fc5-af2d-081406dbb659 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:24.241699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bdc20-d533-4fc5-af2d-081406dbb659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:24.259100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bdc20-d533-4fc5-af2d-081406dbb659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:24.263039 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bdc20-d533-4fc5-af2d-081406dbb659 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:24.290562 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 932/1887] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:24.301101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb24e5-7c49-4046-8795-0a5681d8f122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:24.302862 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb24e5-7c49-4046-8795-0a5681d8f122 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:24.315409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 956/1888] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:24.415608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bdc20-d533-4fc5-af2d-081406dbb659 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:24.419383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f21bdc20-d533-4fc5-af2d-081406dbb659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:24.456342 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bdc20-d533-4fc5-af2d-081406dbb659 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:24.472565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:24.474975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:24.475265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:24.475542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] PATH_INFO: `/v3/projects/487246a64bfb4799823bf433b772fd5a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:24.480431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=487246a64bfb4799823bf433b772fd5a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:24.483691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:24.590783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '487246a64bfb4799823bf433b772fd5a', 'request_id': 'req-3f897474-6044-494b-8269-4be0634ffe9e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.607267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '487246a64bfb4799823bf433b772fd5a', 'request_id': 'req-3f897474-6044-494b-8269-4be0634ffe9e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.617907 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '487246a64bfb4799823bf433b772fd5a', 'request_id': 'req-3f897474-6044-494b-8269-4be0634ffe9e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.636800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] The token cache is being invalidate because project 487246a64bfb4799823bf433b772fd5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:24.637012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.639467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.641875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 933/1889] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:24 2022] DELETE /identity/v3/projects/487246a64bfb4799823bf433b772fd5a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:24.645838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb24e5-7c49-4046-8795-0a5681d8f122 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:24.659236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb24e5-7c49-4046-8795-0a5681d8f122 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:24.848582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:24.850079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:24.872195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:24.872890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:24.873166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:24.873686 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] PATH_INFO: `/v3/users/d880d63c3185437b9d464bb889feb055` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:24.880419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d880d63c3185437b9d464bb889feb055)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:24.882637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:24.955032 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f897474-6044-494b-8269-4be0634ffe9e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:24.961649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:24.962518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:24.962978 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:24.963407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] PATH_INFO: `/v3/projects/d53fb79a0070496e92fe23e1c0da1f7d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:24.964621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] RBAC: Authorizing `identity:delete_project(project_id=d53fb79a0070496e92fe23e1c0da1f7d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:24.966648 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:24.966963 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd880d63c3185437b9d464bb889feb055'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.967617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd880d63c3185437b9d464bb889feb055'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.968225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd880d63c3185437b9d464bb889feb055'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.983306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd880d63c3185437b9d464bb889feb055'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.987561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:24.992602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd880d63c3185437b9d464bb889feb055'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.006728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 957/1890] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:24 2022] DELETE /identity/v3/users/d880d63c3185437b9d464bb889feb055 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:25.031742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd53fb79a0070496e92fe23e1c0da1f7d', 'request_id': 'req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.042807 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd53fb79a0070496e92fe23e1c0da1f7d', 'request_id': 'req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.051159 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd53fb79a0070496e92fe23e1c0da1f7d', 'request_id': 'req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.066576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] The token cache is being invalidate because project d53fb79a0070496e92fe23e1c0da1f7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:25.067068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.067996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.073225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 934/1891] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:24 2022] DELETE /identity/v3/projects/d53fb79a0070496e92fe23e1c0da1f7d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:25.074786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:25.076441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:25.122105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:25.123780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:25.135773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:25.184841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd2695-f2d4-42c7-badc-8091b8775263 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:25.226461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8fd2e-afdb-4e6b-906b-9a6aefece0da admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:25.239953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:25.240895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:25.241340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:25.241767 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] PATH_INFO: `/v3/projects/eab3f01e676c44e385a35fa2d2698923` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:25.243701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab3f01e676c44e385a35fa2d2698923)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:25.246634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:25.287800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:25.290016 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:25.292092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:25.292391 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] PATH_INFO: `/v3/users/124ebd7bae2e4c10ac572c3d898dc555` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:25.295202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=124ebd7bae2e4c10ac572c3d898dc555)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:25.299201 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:25.329862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab3f01e676c44e385a35fa2d2698923', 'request_id': 'req-eb713ac7-637e-4bc3-a286-377d3ce0bf18'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.343747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab3f01e676c44e385a35fa2d2698923', 'request_id': 'req-eb713ac7-637e-4bc3-a286-377d3ce0bf18'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.353506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab3f01e676c44e385a35fa2d2698923', 'request_id': 'req-eb713ac7-637e-4bc3-a286-377d3ce0bf18'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.370558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] The token cache is being invalidate because project eab3f01e676c44e385a35fa2d2698923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:25.373816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.382004 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.390085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 935/1892] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:25 2022] DELETE /identity/v3/projects/eab3f01e676c44e385a35fa2d2698923 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:25.393393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:25.398338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:25.400522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124ebd7bae2e4c10ac572c3d898dc555'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.402390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124ebd7bae2e4c10ac572c3d898dc555'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.403680 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124ebd7bae2e4c10ac572c3d898dc555'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.435152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124ebd7bae2e4c10ac572c3d898dc555'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.457483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124ebd7bae2e4c10ac572c3d898dc555'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.487041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 958/1893] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:25 2022] DELETE /identity/v3/users/124ebd7bae2e4c10ac572c3d898dc555 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:25.488166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:25.492228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:25.496340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:25.601436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb713ac7-637e-4bc3-a286-377d3ce0bf18 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:25.696060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba12bce-27bf-46ca-b18e-bd3abbcb1f4c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:25.705443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc5276-a586-4b84-9412-22f42d795120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:25.706503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5276-a586-4b84-9412-22f42d795120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:25.706841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9900d4-a68c-481f-8f0d-e5f8dc429348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:25.707262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5276-a586-4b84-9412-22f42d795120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:25.707550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9900d4-a68c-481f-8f0d-e5f8dc429348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:25.707827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9900d4-a68c-481f-8f0d-e5f8dc429348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:25.709067 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5276-a586-4b84-9412-22f42d795120 admin admin] PATH_INFO: `/v3/projects/740ce760f8d94a8caec7ab19c0d12a4c` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:25.709276 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9900d4-a68c-481f-8f0d-e5f8dc429348 admin admin] PATH_INFO: `/v3/users/44f86c88f774400abf4307faae26d31d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:25.710754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9900d4-a68c-481f-8f0d-e5f8dc429348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f86c88f774400abf4307faae26d31d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:25.711110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc5276-a586-4b84-9412-22f42d795120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=740ce760f8d94a8caec7ab19c0d12a4c)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:25.714346 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc5276-a586-4b84-9412-22f42d795120 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:25.718098 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:25.718960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9900d4-a68c-481f-8f0d-e5f8dc429348 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:25.836174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9dbc5276-a586-4b84-9412-22f42d795120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '740ce760f8d94a8caec7ab19c0d12a4c', 'request_id': 'req-9dbc5276-a586-4b84-9412-22f42d795120'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.846638 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ad9900d4-a68c-481f-8f0d-e5f8dc429348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f86c88f774400abf4307faae26d31d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.848704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ad9900d4-a68c-481f-8f0d-e5f8dc429348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f86c88f774400abf4307faae26d31d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.850507 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9dbc5276-a586-4b84-9412-22f42d795120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '740ce760f8d94a8caec7ab19c0d12a4c', 'request_id': 'req-9dbc5276-a586-4b84-9412-22f42d795120'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.850905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ad9900d4-a68c-481f-8f0d-e5f8dc429348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f86c88f774400abf4307faae26d31d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.859320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9dbc5276-a586-4b84-9412-22f42d795120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '740ce760f8d94a8caec7ab19c0d12a4c', 'request_id': 'req-9dbc5276-a586-4b84-9412-22f42d795120'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.875020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9dbc5276-a586-4b84-9412-22f42d795120 admin admin] The token cache is being invalidate because project 740ce760f8d94a8caec7ab19c0d12a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:25.875457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9dbc5276-a586-4b84-9412-22f42d795120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.876300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9dbc5276-a586-4b84-9412-22f42d795120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.881742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 959/1894] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:25 2022] DELETE /identity/v3/projects/740ce760f8d94a8caec7ab19c0d12a4c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:25.881910 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ad9900d4-a68c-481f-8f0d-e5f8dc429348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f86c88f774400abf4307faae26d31d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.883586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0431e48-2367-4146-8567-df41839e7ff4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:25.883981 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0431e48-2367-4146-8567-df41839e7ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:25.884383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0431e48-2367-4146-8567-df41839e7ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:25.906116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ad9900d4-a68c-481f-8f0d-e5f8dc429348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f86c88f774400abf4307faae26d31d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.912060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a0431e48-2367-4146-8567-df41839e7ff4 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:25.913538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a0431e48-2367-4146-8567-df41839e7ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:25.936025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 936/1895] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:25 2022] DELETE /identity/v3/users/44f86c88f774400abf4307faae26d31d => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:26.066077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 960/1896] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:26.067804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6befda2-5be7-4c55-b4e6-cd2b64f16c93 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.068098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6befda2-5be7-4c55-b4e6-cd2b64f16c93 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.068366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6befda2-5be7-4c55-b4e6-cd2b64f16c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.115951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b6befda2-5be7-4c55-b4e6-cd2b64f16c93 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:26.117657 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b6befda2-5be7-4c55-b4e6-cd2b64f16c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:26.212905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:26.240528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 937/1897] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:26.243644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0431e48-2367-4146-8567-df41839e7ff4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.261072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0431e48-2367-4146-8567-df41839e7ff4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.283787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a4b78-2f11-44f4-8080-001291638ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.286837 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a4b78-2f11-44f4-8080-001291638ccd admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.287166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a4b78-2f11-44f4-8080-001291638ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.287456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a4b78-2f11-44f4-8080-001291638ccd admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.288143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a4b78-2f11-44f4-8080-001291638ccd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:26.289059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a4b78-2f11-44f4-8080-001291638ccd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.292080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a4b78-2f11-44f4-8080-001291638ccd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.327088 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 961/1898] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:33:26.328945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6befda2-5be7-4c55-b4e6-cd2b64f16c93 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.340684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6befda2-5be7-4c55-b4e6-cd2b64f16c93 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.347307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e46931-3903-4add-8aaf-359546fbc116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.347921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e46931-3903-4add-8aaf-359546fbc116 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.348302 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e46931-3903-4add-8aaf-359546fbc116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.348605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e46931-3903-4add-8aaf-359546fbc116 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.349337 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e46931-3903-4add-8aaf-359546fbc116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.351855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e46931-3903-4add-8aaf-359546fbc116 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.400793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a4b78-2f11-44f4-8080-001291638ccd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.402990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3d9a4b78-2f11-44f4-8080-001291638ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:26.416112 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 938/1899] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:26 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:26.417282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e46931-3903-4add-8aaf-359546fbc116 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.429843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a4b78-2f11-44f4-8080-001291638ccd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.494413 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e46931-3903-4add-8aaf-359546fbc116 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.499863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b8f8b-4456-465f-8253-eedea8e4ab37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.500476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b8f8b-4456-465f-8253-eedea8e4ab37 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.500899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b8f8b-4456-465f-8253-eedea8e4ab37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.501321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b8f8b-4456-465f-8253-eedea8e4ab37 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.501999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b8f8b-4456-465f-8253-eedea8e4ab37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.504375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b8f8b-4456-465f-8253-eedea8e4ab37 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.536261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 939/1900] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:26 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:26.539006 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b8f8b-4456-465f-8253-eedea8e4ab37 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.540864 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d26936-6e4c-4531-a6b7-74e2223a2279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.541586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d26936-6e4c-4531-a6b7-74e2223a2279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.541868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d26936-6e4c-4531-a6b7-74e2223a2279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.542140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d26936-6e4c-4531-a6b7-74e2223a2279 admin admin] PATH_INFO: `/v3/projects/aa5615029a42483d88e9298947d37c41` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.546107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d26936-6e4c-4531-a6b7-74e2223a2279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa5615029a42483d88e9298947d37c41)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.548180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d26936-6e4c-4531-a6b7-74e2223a2279 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.551671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b8f8b-4456-465f-8253-eedea8e4ab37 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.557886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af253bc2-a33f-4267-889f-19fea9b5032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.558664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af253bc2-a33f-4267-889f-19fea9b5032d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.558948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af253bc2-a33f-4267-889f-19fea9b5032d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.559812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af253bc2-a33f-4267-889f-19fea9b5032d admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.560561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af253bc2-a33f-4267-889f-19fea9b5032d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.562736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af253bc2-a33f-4267-889f-19fea9b5032d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.580946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 940/1901] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:26.580946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2373fc1-2aa4-4ea9-a14f-a069db0a2e7b None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.580946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2373fc1-2aa4-4ea9-a14f-a069db0a2e7b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.581313 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2373fc1-2aa4-4ea9-a14f-a069db0a2e7b None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.583199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 941/1902] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:33:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:33:26.583971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2373fc1-2aa4-4ea9-a14f-a069db0a2e7b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.598573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:26.604723 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2373fc1-2aa4-4ea9-a14f-a069db0a2e7b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.613262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc673c-1713-4319-93b6-5502ea616ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.614111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc673c-1713-4319-93b6-5502ea616ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.614486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc673c-1713-4319-93b6-5502ea616ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.614857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc673c-1713-4319-93b6-5502ea616ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.615524 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc673c-1713-4319-93b6-5502ea616ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.617885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc673c-1713-4319-93b6-5502ea616ea0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.633165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 942/1903] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:26.635605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc673c-1713-4319-93b6-5502ea616ea0 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.654813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc673c-1713-4319-93b6-5502ea616ea0 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.659285 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-76d26936-6e4c-4531-a6b7-74e2223a2279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa5615029a42483d88e9298947d37c41', 'request_id': 'req-76d26936-6e4c-4531-a6b7-74e2223a2279'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.671438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-76d26936-6e4c-4531-a6b7-74e2223a2279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa5615029a42483d88e9298947d37c41', 'request_id': 'req-76d26936-6e4c-4531-a6b7-74e2223a2279'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.687664 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-76d26936-6e4c-4531-a6b7-74e2223a2279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa5615029a42483d88e9298947d37c41', 'request_id': 'req-76d26936-6e4c-4531-a6b7-74e2223a2279'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.707824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-76d26936-6e4c-4531-a6b7-74e2223a2279 admin admin] The token cache is being invalidate because project aa5615029a42483d88e9298947d37c41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:26.708390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-76d26936-6e4c-4531-a6b7-74e2223a2279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.709037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-76d26936-6e4c-4531-a6b7-74e2223a2279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:26.711363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 962/1904] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:26 2022] DELETE /identity/v3/projects/aa5615029a42483d88e9298947d37c41 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:26.713075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216caa-4e2f-4115-a020-517c1ddddac1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.713603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216caa-4e2f-4115-a020-517c1ddddac1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.713970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216caa-4e2f-4115-a020-517c1ddddac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.746131 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6d216caa-4e2f-4115-a020-517c1ddddac1 None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:26.747563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6d216caa-4e2f-4115-a020-517c1ddddac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:26.757734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2afc1-e92b-4922-8d49-4aa25799a983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.758776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2afc1-e92b-4922-8d49-4aa25799a983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.759138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2afc1-e92b-4922-8d49-4aa25799a983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.759495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2afc1-e92b-4922-8d49-4aa25799a983 admin admin] PATH_INFO: `/v3/projects/5cc5433a250544a1b3fbd6014605ad37/users/0c74d57f2e824c65a8daf20e0078cc10/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.794654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2afc1-e92b-4922-8d49-4aa25799a983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc5433a250544a1b3fbd6014605ad37, user_id=0c74d57f2e824c65a8daf20e0078cc10, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.798896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2afc1-e92b-4922-8d49-4aa25799a983 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.830915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 943/1905] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:26 2022] PUT /identity/v3/projects/5cc5433a250544a1b3fbd6014605ad37/users/0c74d57f2e824c65a8daf20e0078cc10/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:26.830915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2afc1-e92b-4922-8d49-4aa25799a983 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.832629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-02c2afc1-e92b-4922-8d49-4aa25799a983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:26.891105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:26.894553 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 963/1906] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:26.896313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f5adf-710b-43a6-aa0e-41cb8998a5b5 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.897045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f5adf-710b-43a6-aa0e-41cb8998a5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.897511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f5adf-710b-43a6-aa0e-41cb8998a5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.929736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ec6f5adf-710b-43a6-aa0e-41cb8998a5b5 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:26.931354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ec6f5adf-710b-43a6-aa0e-41cb8998a5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.015519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2afc1-e92b-4922-8d49-4aa25799a983 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.021878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.022696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.022986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.023278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce admin admin] PATH_INFO: `/v3/projects/943fc2d5359341e99357c7d27145f754` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.024607 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=943fc2d5359341e99357c7d27145f754)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.028130 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.051142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 964/1907] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:27.051142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a7f32-c1ec-42a4-8146-07329cbc4fa9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.051142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a7f32-c1ec-42a4-8146-07329cbc4fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.051142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a7f32-c1ec-42a4-8146-07329cbc4fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.131415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ac1a7f32-c1ec-42a4-8146-07329cbc4fa9 None None] MFA Rules not processed for user `0c74d57f2e824c65a8daf20e0078cc10`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:27.148449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ac1a7f32-c1ec-42a4-8146-07329cbc4fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.152560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '943fc2d5359341e99357c7d27145f754', 'request_id': 'req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:27.177130 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '943fc2d5359341e99357c7d27145f754', 'request_id': 'req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:27.185926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '943fc2d5359341e99357c7d27145f754', 'request_id': 'req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:27.207966 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce admin admin] The token cache is being invalidate because project 943fc2d5359341e99357c7d27145f754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:27.208311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:27.209158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1be8d3a8-2528-46fd-967a-bdb763d0e2ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:27.211281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 944/1908] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:26 2022] DELETE /identity/v3/projects/943fc2d5359341e99357c7d27145f754 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:27.212763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e765e-0045-496f-b6b6-bb17040ed22d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.213165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e765e-0045-496f-b6b6-bb17040ed22d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.213554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e765e-0045-496f-b6b6-bb17040ed22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.231845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:27.252455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-df1e765e-0045-496f-b6b6-bb17040ed22d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:27.254470 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-df1e765e-0045-496f-b6b6-bb17040ed22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.308425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 965/1909] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:27.309571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a7f32-c1ec-42a4-8146-07329cbc4fa9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.310991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ac1a7f32-c1ec-42a4-8146-07329cbc4fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.344815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 945/1910] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:27.346642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6e8f2-dd41-477d-86d7-c259979d7455 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.347087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6e8f2-dd41-477d-86d7-c259979d7455 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.347599 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6e8f2-dd41-477d-86d7-c259979d7455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.383684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-58b6e8f2-dd41-477d-86d7-c259979d7455 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:27.384950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-58b6e8f2-dd41-477d-86d7-c259979d7455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.403732 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 946/1911] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:27.404351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6e8f2-dd41-477d-86d7-c259979d7455 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.411378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-58b6e8f2-dd41-477d-86d7-c259979d7455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.426369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a7f32-c1ec-42a4-8146-07329cbc4fa9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.427857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6e8f2-dd41-477d-86d7-c259979d7455 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.434113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61882681-513e-4da7-bca3-f7a3e9b697e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.434740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61882681-513e-4da7-bca3-f7a3e9b697e2 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.435015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61882681-513e-4da7-bca3-f7a3e9b697e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.435287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61882681-513e-4da7-bca3-f7a3e9b697e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.435882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f712d99-8fcc-4527-b473-36decc35cad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.436693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f712d99-8fcc-4527-b473-36decc35cad9 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.436885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f712d99-8fcc-4527-b473-36decc35cad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.437160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f712d99-8fcc-4527-b473-36decc35cad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.437449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-61882681-513e-4da7-bca3-f7a3e9b697e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.437796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f712d99-8fcc-4527-b473-36decc35cad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.441122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f712d99-8fcc-4527-b473-36decc35cad9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.455028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61882681-513e-4da7-bca3-f7a3e9b697e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.456628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61882681-513e-4da7-bca3-f7a3e9b697e2 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.475343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 947/1912] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:27.476478 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f712d99-8fcc-4527-b473-36decc35cad9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.477563 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 966/1913] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:27.480483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7bf0b-381e-40b0-9c50-31d6727ef279 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.480812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7bf0b-381e-40b0-9c50-31d6727ef279 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.481200 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7bf0b-381e-40b0-9c50-31d6727ef279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.510326 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-26d7bf0b-381e-40b0-9c50-31d6727ef279 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:27.511821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-26d7bf0b-381e-40b0-9c50-31d6727ef279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.521109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:27.568745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f712d99-8fcc-4527-b473-36decc35cad9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.575682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2044880-4a84-47ce-9b51-4bb101053b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.576454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2044880-4a84-47ce-9b51-4bb101053b09 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.576822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2044880-4a84-47ce-9b51-4bb101053b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.577508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2044880-4a84-47ce-9b51-4bb101053b09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.578028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2044880-4a84-47ce-9b51-4bb101053b09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:27.578628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2044880-4a84-47ce-9b51-4bb101053b09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.580254 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2044880-4a84-47ce-9b51-4bb101053b09 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.603998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 948/1914] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:27.605174 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2044880-4a84-47ce-9b51-4bb101053b09 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.616152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 967/1915] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:27.617700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7bf0b-381e-40b0-9c50-31d6727ef279 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.630234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7bf0b-381e-40b0-9c50-31d6727ef279 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.640492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2044880-4a84-47ce-9b51-4bb101053b09 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.642634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760baca6-15ff-4599-98dd-0ab272d4cf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.644570 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760baca6-15ff-4599-98dd-0ab272d4cf42 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.645251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760baca6-15ff-4599-98dd-0ab272d4cf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.645884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760baca6-15ff-4599-98dd-0ab272d4cf42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.646716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760baca6-15ff-4599-98dd-0ab272d4cf42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.647113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086c3a5-7994-4d24-8edb-2a4df6a1c15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.647730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086c3a5-7994-4d24-8edb-2a4df6a1c15c admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.648009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086c3a5-7994-4d24-8edb-2a4df6a1c15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.648277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086c3a5-7994-4d24-8edb-2a4df6a1c15c admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.649228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086c3a5-7994-4d24-8edb-2a4df6a1c15c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.650468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760baca6-15ff-4599-98dd-0ab272d4cf42 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.651681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086c3a5-7994-4d24-8edb-2a4df6a1c15c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.691991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 949/1916] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/users => generated 290 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:27.693807 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086c3a5-7994-4d24-8edb-2a4df6a1c15c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.716861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086c3a5-7994-4d24-8edb-2a4df6a1c15c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.717780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 968/1917] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:27.719583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760baca6-15ff-4599-98dd-0ab272d4cf42 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.795353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb17f06-f0f1-4879-b3d7-7b3ec81a7ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.795877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb17f06-f0f1-4879-b3d7-7b3ec81a7ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.796225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb17f06-f0f1-4879-b3d7-7b3ec81a7ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.796565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb17f06-f0f1-4879-b3d7-7b3ec81a7ba2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.797374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb17f06-f0f1-4879-b3d7-7b3ec81a7ba2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:27.797816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb17f06-f0f1-4879-b3d7-7b3ec81a7ba2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.799598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb17f06-f0f1-4879-b3d7-7b3ec81a7ba2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.804008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:27.823548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 950/1918] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:27.828975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb17f06-f0f1-4879-b3d7-7b3ec81a7ba2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.840467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb17f06-f0f1-4879-b3d7-7b3ec81a7ba2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.847097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead9faa-1948-4bc4-9c08-30435208227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.847339 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead9faa-1948-4bc4-9c08-30435208227c admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.847513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead9faa-1948-4bc4-9c08-30435208227c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.848136 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead9faa-1948-4bc4-9c08-30435208227c admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.849670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead9faa-1948-4bc4-9c08-30435208227c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.851413 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead9faa-1948-4bc4-9c08-30435208227c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.871300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760baca6-15ff-4599-98dd-0ab272d4cf42 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.880222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a1567-2aa2-4cbc-ac0e-daece665070b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.881001 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a1567-2aa2-4cbc-ac0e-daece665070b admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.881283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a1567-2aa2-4cbc-ac0e-daece665070b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.881550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a1567-2aa2-4cbc-ac0e-daece665070b admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.882254 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a1567-2aa2-4cbc-ac0e-daece665070b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.883960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a1567-2aa2-4cbc-ac0e-daece665070b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.901090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 969/1919] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:27.902773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 951/1920] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/users => generated 292 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:27.905705 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a1567-2aa2-4cbc-ac0e-daece665070b admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.906901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead9faa-1948-4bc4-9c08-30435208227c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.921828 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a1567-2aa2-4cbc-ac0e-daece665070b admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.923677 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead9faa-1948-4bc4-9c08-30435208227c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.929669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e901e2-8261-4a8f-822b-9a2025bf6a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.930103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e901e2-8261-4a8f-822b-9a2025bf6a1e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.930893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e901e2-8261-4a8f-822b-9a2025bf6a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.931583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e901e2-8261-4a8f-822b-9a2025bf6a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.932537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e901e2-8261-4a8f-822b-9a2025bf6a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.935516 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e901e2-8261-4a8f-822b-9a2025bf6a1e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.943761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733faa9-a485-4cc4-b7bd-0fe6e4817c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.945042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733faa9-a485-4cc4-b7bd-0fe6e4817c27 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.945331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733faa9-a485-4cc4-b7bd-0fe6e4817c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.945602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733faa9-a485-4cc4-b7bd-0fe6e4817c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.946247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733faa9-a485-4cc4-b7bd-0fe6e4817c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.949358 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733faa9-a485-4cc4-b7bd-0fe6e4817c27 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.965940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 952/1921] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:27.968572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e901e2-8261-4a8f-822b-9a2025bf6a1e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.995930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e901e2-8261-4a8f-822b-9a2025bf6a1e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.003551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 970/1922] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:28.004378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733faa9-a485-4cc4-b7bd-0fe6e4817c27 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.004593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885deb1c-311d-4eeb-b886-897e8d25415f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.006595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885deb1c-311d-4eeb-b886-897e8d25415f admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.009719 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885deb1c-311d-4eeb-b886-897e8d25415f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.011003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885deb1c-311d-4eeb-b886-897e8d25415f admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.012856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885deb1c-311d-4eeb-b886-897e8d25415f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.015023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885deb1c-311d-4eeb-b886-897e8d25415f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.033362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 953/1923] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:28.034365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885deb1c-311d-4eeb-b886-897e8d25415f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.049682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885deb1c-311d-4eeb-b886-897e8d25415f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.084130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733faa9-a485-4cc4-b7bd-0fe6e4817c27 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.090470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49837d6e-d4a4-4b43-9b98-3cc36b190d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.091183 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49837d6e-d4a4-4b43-9b98-3cc36b190d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.091469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49837d6e-d4a4-4b43-9b98-3cc36b190d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.091748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49837d6e-d4a4-4b43-9b98-3cc36b190d6d admin admin] PATH_INFO: `/v3/projects/e9eb589e553e4f32858e8da366569c04/users/fa6ddfcf42be46e391132a60d301392c/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.130456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49837d6e-d4a4-4b43-9b98-3cc36b190d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9eb589e553e4f32858e8da366569c04, user_id=fa6ddfcf42be46e391132a60d301392c, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.133637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49837d6e-d4a4-4b43-9b98-3cc36b190d6d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.153989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 971/1924] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:28 2022] PUT /identity/v3/projects/e9eb589e553e4f32858e8da366569c04/users/fa6ddfcf42be46e391132a60d301392c/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:28.154940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49837d6e-d4a4-4b43-9b98-3cc36b190d6d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.161106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bbf56-c407-45bb-b35d-de6dbddb5b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.162250 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bbf56-c407-45bb-b35d-de6dbddb5b4f admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.162532 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bbf56-c407-45bb-b35d-de6dbddb5b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.162804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bbf56-c407-45bb-b35d-de6dbddb5b4f admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.163443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bbf56-c407-45bb-b35d-de6dbddb5b4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.166235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bbf56-c407-45bb-b35d-de6dbddb5b4f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.179027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:28.204552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 954/1925] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:28 2022] POST /identity/v3/users => generated 295 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:28.204552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5b7ad-9f8e-44ca-b696-87056cb9e26f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.204552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5b7ad-9f8e-44ca-b696-87056cb9e26f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.204552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5b7ad-9f8e-44ca-b696-87056cb9e26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.244213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f7e5b7ad-9f8e-44ca-b696-87056cb9e26f None None] MFA Rules not processed for user `fa6ddfcf42be46e391132a60d301392c`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:28.245597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f7e5b7ad-9f8e-44ca-b696-87056cb9e26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:28.255018 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49837d6e-d4a4-4b43-9b98-3cc36b190d6d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.261585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae18d7-2f03-41df-bffc-72e938fd33db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.261758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae18d7-2f03-41df-bffc-72e938fd33db admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.262069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae18d7-2f03-41df-bffc-72e938fd33db admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.262412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae18d7-2f03-41df-bffc-72e938fd33db admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.263129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae18d7-2f03-41df-bffc-72e938fd33db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.265227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae18d7-2f03-41df-bffc-72e938fd33db admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.276396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 972/1926] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:28.277322 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae18d7-2f03-41df-bffc-72e938fd33db admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.288345 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae18d7-2f03-41df-bffc-72e938fd33db admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.293399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247b8b4-0e41-4a67-91e7-98b475e1a09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.294640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247b8b4-0e41-4a67-91e7-98b475e1a09a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.294940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247b8b4-0e41-4a67-91e7-98b475e1a09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.295226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247b8b4-0e41-4a67-91e7-98b475e1a09a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.296207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247b8b4-0e41-4a67-91e7-98b475e1a09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.299072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247b8b4-0e41-4a67-91e7-98b475e1a09a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.313984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 973/1927] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:28.313984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247b8b4-0e41-4a67-91e7-98b475e1a09a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.313984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c247b8b4-0e41-4a67-91e7-98b475e1a09a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:28.327122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 955/1928] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:28.327400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247b8b4-0e41-4a67-91e7-98b475e1a09a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.329735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5b7ad-9f8e-44ca-b696-87056cb9e26f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.334593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.335266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.335549 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.335858 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] PATH_INFO: `/v3/users/c6cac6d8b2b9453a879145131ccfc7b6` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.337005 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6cac6d8b2b9453a879145131ccfc7b6)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.339105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.349619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5b7ad-9f8e-44ca-b696-87056cb9e26f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.353358 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:28.354895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa48637-877b-42f4-a8f9-ef7799802973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.355570 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa48637-877b-42f4-a8f9-ef7799802973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.355850 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa48637-877b-42f4-a8f9-ef7799802973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.356122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa48637-877b-42f4-a8f9-ef7799802973 admin admin] PATH_INFO: `/v3/projects/ecc86640a4c04815b3f59912b70b01e7/users/4a19cc9ba70d4c95a7c2bcdbf2de8d81/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.374847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa48637-877b-42f4-a8f9-ef7799802973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc86640a4c04815b3f59912b70b01e7, user_id=4a19cc9ba70d4c95a7c2bcdbf2de8d81, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.377928 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa48637-877b-42f4-a8f9-ef7799802973 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.394995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 956/1929] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:28 2022] PUT /identity/v3/projects/ecc86640a4c04815b3f59912b70b01e7/users/4a19cc9ba70d4c95a7c2bcdbf2de8d81/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:28.396151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa48637-877b-42f4-a8f9-ef7799802973 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.418632 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6cac6d8b2b9453a879145131ccfc7b6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:28.419400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6cac6d8b2b9453a879145131ccfc7b6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:28.432139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6cac6d8b2b9453a879145131ccfc7b6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:28.447545 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6cac6d8b2b9453a879145131ccfc7b6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:28.457900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6cac6d8b2b9453a879145131ccfc7b6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:28.474161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 974/1930] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:28 2022] DELETE /identity/v3/users/c6cac6d8b2b9453a879145131ccfc7b6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:28.475280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.478198 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:28.486079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa48637-877b-42f4-a8f9-ef7799802973 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.555296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1eaa8-f594-4b9e-9491-480ee9253697 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.557202 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85096a-ed6c-4e0b-918b-bc329a7e50ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.557835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85096a-ed6c-4e0b-918b-bc329a7e50ae admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.558234 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85096a-ed6c-4e0b-918b-bc329a7e50ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.558518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85096a-ed6c-4e0b-918b-bc329a7e50ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.559174 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85096a-ed6c-4e0b-918b-bc329a7e50ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.560272 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb987f9-3c66-441a-b350-d0eccfc1e9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.560994 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb987f9-3c66-441a-b350-d0eccfc1e9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.561112 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85096a-ed6c-4e0b-918b-bc329a7e50ae admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.561356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb987f9-3c66-441a-b350-d0eccfc1e9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.561700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb987f9-3c66-441a-b350-d0eccfc1e9f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.562464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb987f9-3c66-441a-b350-d0eccfc1e9f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.564717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb987f9-3c66-441a-b350-d0eccfc1e9f4 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.567767 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:28.573540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 957/1931] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:28.577979 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3694ea-c16f-49fc-bf50-5e935ee4cda7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.579127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3694ea-c16f-49fc-bf50-5e935ee4cda7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.579466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3694ea-c16f-49fc-bf50-5e935ee4cda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.598746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 975/1932] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:28 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:28.598746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f432c-9f9f-402c-87d1-d99c75bbe250 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.598746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f432c-9f9f-402c-87d1-d99c75bbe250 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.598746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f432c-9f9f-402c-87d1-d99c75bbe250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.615423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-cc3694ea-c16f-49fc-bf50-5e935ee4cda7 None None] MFA Rules not processed for user `4a19cc9ba70d4c95a7c2bcdbf2de8d81`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:28.616634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cc3694ea-c16f-49fc-bf50-5e935ee4cda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:28.617546 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-796f432c-9f9f-402c-87d1-d99c75bbe250 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:28.618724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-796f432c-9f9f-402c-87d1-d99c75bbe250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:28.687530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 958/1933] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:28.687530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3694ea-c16f-49fc-bf50-5e935ee4cda7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.687530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cc3694ea-c16f-49fc-bf50-5e935ee4cda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:28.690637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 976/1934] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:28.691876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f432c-9f9f-402c-87d1-d99c75bbe250 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.704312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3694ea-c16f-49fc-bf50-5e935ee4cda7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.709853 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88482b7-059e-4234-b06c-99c0f5c74fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.710103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f432c-9f9f-402c-87d1-d99c75bbe250 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.710517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88482b7-059e-4234-b06c-99c0f5c74fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.710795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88482b7-059e-4234-b06c-99c0f5c74fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.711054 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88482b7-059e-4234-b06c-99c0f5c74fc8 admin admin] PATH_INFO: `/v3/projects/40fecd21b678483c847e5207e1677db5/users/eba485c920b44459bb9e652705a27b57/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.716573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6c67e-c146-4894-91bc-2e6b7049732d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.717225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6c67e-c146-4894-91bc-2e6b7049732d admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.717508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6c67e-c146-4894-91bc-2e6b7049732d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.717783 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6c67e-c146-4894-91bc-2e6b7049732d admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.718408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6c67e-c146-4894-91bc-2e6b7049732d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.720496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6c67e-c146-4894-91bc-2e6b7049732d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.735820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88482b7-059e-4234-b06c-99c0f5c74fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fecd21b678483c847e5207e1677db5, user_id=eba485c920b44459bb9e652705a27b57, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.738766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88482b7-059e-4234-b06c-99c0f5c74fc8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.739259 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:28.748291 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 977/1935] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:28 2022] POST /identity/v3/users => generated 295 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:28.749564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6c67e-c146-4894-91bc-2e6b7049732d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.753559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ece6c67e-c146-4894-91bc-2e6b7049732d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:28.754703 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 959/1936] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:28 2022] PUT /identity/v3/projects/40fecd21b678483c847e5207e1677db5/users/eba485c920b44459bb9e652705a27b57/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:28.755667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88482b7-059e-4234-b06c-99c0f5c74fc8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.758052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f88482b7-059e-4234-b06c-99c0f5c74fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:28.822466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88482b7-059e-4234-b06c-99c0f5c74fc8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.828027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c06d1-a109-495e-912d-f062cc4d8b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.828631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c06d1-a109-495e-912d-f062cc4d8b28 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.829174 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c06d1-a109-495e-912d-f062cc4d8b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.829773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c06d1-a109-495e-912d-f062cc4d8b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.834283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6c67e-c146-4894-91bc-2e6b7049732d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.890578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c284e-b8d1-4a86-a609-e0a07f7de5fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.891211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c284e-b8d1-4a86-a609-e0a07f7de5fc service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.891685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c284e-b8d1-4a86-a609-e0a07f7de5fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.892095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c284e-b8d1-4a86-a609-e0a07f7de5fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.911798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c06d1-a109-495e-912d-f062cc4d8b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.913301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c06d1-a109-495e-912d-f062cc4d8b28 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.923716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 960/1937] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:28.924464 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c06d1-a109-495e-912d-f062cc4d8b28 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.933995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c06d1-a109-495e-912d-f062cc4d8b28 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.939282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30b392-a2dc-40a0-badb-b7b47f051fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:28.940213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30b392-a2dc-40a0-badb-b7b47f051fca admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.940440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30b392-a2dc-40a0-badb-b7b47f051fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.940759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30b392-a2dc-40a0-badb-b7b47f051fca admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.941390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30b392-a2dc-40a0-badb-b7b47f051fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.942975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30b392-a2dc-40a0-badb-b7b47f051fca admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.953944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 961/1938] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:28.955168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a9626-dd65-443f-a720-6d615895bf54 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:28.955277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a9626-dd65-443f-a720-6d615895bf54 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:28.955571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a9626-dd65-443f-a720-6d615895bf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:28.959185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c284e-b8d1-4a86-a609-e0a07f7de5fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:28.960699 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c284e-b8d1-4a86-a609-e0a07f7de5fc service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:28.964237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:28.970610 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 978/1939] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:28.971457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c284e-b8d1-4a86-a609-e0a07f7de5fc service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:28.990284 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c284e-b8d1-4a86-a609-e0a07f7de5fc service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:28.999362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45196b-d071-4949-b2d4-a6a3d6d35fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.000032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45196b-d071-4949-b2d4-a6a3d6d35fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.000307 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45196b-d071-4949-b2d4-a6a3d6d35fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.000621 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45196b-d071-4949-b2d4-a6a3d6d35fc0 admin admin] PATH_INFO: `/v3/projects/1eefa17e033d452988d2c0d6630d53ca/users/0eba5721cc45492e9f448d77abb490a7/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.006448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f86a9626-dd65-443f-a720-6d615895bf54 None None] MFA Rules not processed for user `eba485c920b44459bb9e652705a27b57`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:29.007784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f86a9626-dd65-443f-a720-6d615895bf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.017684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45196b-d071-4949-b2d4-a6a3d6d35fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eefa17e033d452988d2c0d6630d53ca, user_id=0eba5721cc45492e9f448d77abb490a7, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.020432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45196b-d071-4949-b2d4-a6a3d6d35fc0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.034934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 979/1940] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:28 2022] PUT /identity/v3/projects/1eefa17e033d452988d2c0d6630d53ca/users/0eba5721cc45492e9f448d77abb490a7/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:29.037969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45196b-d071-4949-b2d4-a6a3d6d35fc0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.039824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9a45196b-d071-4949-b2d4-a6a3d6d35fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.083668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 962/1941] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:29.084799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a9626-dd65-443f-a720-6d615895bf54 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.096783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a9626-dd65-443f-a720-6d615895bf54 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.100052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:29.121408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45196b-d071-4949-b2d4-a6a3d6d35fc0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.126254 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.126901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.127183 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.127458 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] PATH_INFO: `/v3/users/4a19cc9ba70d4c95a7c2bcdbf2de8d81` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.128637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a19cc9ba70d4c95a7c2bcdbf2de8d81)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.130807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.177565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c072bac-579f-469c-a90f-ddbabb9e3f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.178397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c072bac-579f-469c-a90f-ddbabb9e3f69 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.178724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c072bac-579f-469c-a90f-ddbabb9e3f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.179071 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c072bac-579f-469c-a90f-ddbabb9e3f69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.180145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c072bac-579f-469c-a90f-ddbabb9e3f69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.181468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c072bac-579f-469c-a90f-ddbabb9e3f69 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.192388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 963/1942] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:29.196379 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c072bac-579f-469c-a90f-ddbabb9e3f69 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.196500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a19cc9ba70d4c95a7c2bcdbf2de8d81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.197556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a19cc9ba70d4c95a7c2bcdbf2de8d81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.197785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a19cc9ba70d4c95a7c2bcdbf2de8d81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.198556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4c072bac-579f-469c-a90f-ddbabb9e3f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.212165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c072bac-579f-469c-a90f-ddbabb9e3f69 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.217072 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df71a56-626b-43bb-9e97-713496d093cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.217671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df71a56-626b-43bb-9e97-713496d093cd admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.218084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df71a56-626b-43bb-9e97-713496d093cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.218258 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df71a56-626b-43bb-9e97-713496d093cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.218820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df71a56-626b-43bb-9e97-713496d093cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.221011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df71a56-626b-43bb-9e97-713496d093cd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.224207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a19cc9ba70d4c95a7c2bcdbf2de8d81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.239677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a19cc9ba70d4c95a7c2bcdbf2de8d81'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.252777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 964/1943] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:29 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:29.254058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df71a56-626b-43bb-9e97-713496d093cd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.257121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6df71a56-626b-43bb-9e97-713496d093cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.257327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 980/1944] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:29 2022] DELETE /identity/v3/users/4a19cc9ba70d4c95a7c2bcdbf2de8d81 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:29.258150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.259481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.332078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df71a56-626b-43bb-9e97-713496d093cd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.337417 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d9e54-0e93-45c3-8b0e-566020770a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.338010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d9e54-0e93-45c3-8b0e-566020770a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.338307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d9e54-0e93-45c3-8b0e-566020770a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.338576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d9e54-0e93-45c3-8b0e-566020770a7d admin admin] PATH_INFO: `/v3/projects/1eefa17e033d452988d2c0d6630d53ca/users/0eba5721cc45492e9f448d77abb490a7/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.339125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fbd53-567b-4bb8-8555-9afd853d68ed admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.340687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d9e54-0e93-45c3-8b0e-566020770a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eefa17e033d452988d2c0d6630d53ca, user_id=0eba5721cc45492e9f448d77abb490a7, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.343303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d9e54-0e93-45c3-8b0e-566020770a7d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.343975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cfce3-247d-4354-9a44-31920ce96434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.344980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cfce3-247d-4354-9a44-31920ce96434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.345281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cfce3-247d-4354-9a44-31920ce96434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.345589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cfce3-247d-4354-9a44-31920ce96434 admin admin] PATH_INFO: `/v3/projects/15c5ccea4f69476d8b147b8eef356470` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.347539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cfce3-247d-4354-9a44-31920ce96434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15c5ccea4f69476d8b147b8eef356470)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.349933 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cfce3-247d-4354-9a44-31920ce96434 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.356940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 965/1945] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:29 2022] PUT /identity/v3/projects/1eefa17e033d452988d2c0d6630d53ca/users/0eba5721cc45492e9f448d77abb490a7/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:29.357748 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d9e54-0e93-45c3-8b0e-566020770a7d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.364595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:29.437737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-148cfce3-247d-4354-9a44-31920ce96434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15c5ccea4f69476d8b147b8eef356470', 'request_id': 'req-148cfce3-247d-4354-9a44-31920ce96434'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.451720 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-148cfce3-247d-4354-9a44-31920ce96434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15c5ccea4f69476d8b147b8eef356470', 'request_id': 'req-148cfce3-247d-4354-9a44-31920ce96434'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.461193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-148cfce3-247d-4354-9a44-31920ce96434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15c5ccea4f69476d8b147b8eef356470', 'request_id': 'req-148cfce3-247d-4354-9a44-31920ce96434'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.466045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:29.482216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d9e54-0e93-45c3-8b0e-566020770a7d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.486247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-148cfce3-247d-4354-9a44-31920ce96434 admin admin] The token cache is being invalidate because project 15c5ccea4f69476d8b147b8eef356470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:29.488385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-148cfce3-247d-4354-9a44-31920ce96434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.489842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-148cfce3-247d-4354-9a44-31920ce96434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.493960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 981/1946] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:29 2022] DELETE /identity/v3/projects/15c5ccea4f69476d8b147b8eef356470 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:29.495565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada8745-236b-44b5-9ad2-9540d7492592 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.496051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada8745-236b-44b5-9ad2-9540d7492592 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.496620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada8745-236b-44b5-9ad2-9540d7492592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.524357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9ada8745-236b-44b5-9ad2-9540d7492592 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:29.526390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9ada8745-236b-44b5-9ad2-9540d7492592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.596072 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d135efa-4e89-49fa-94fb-5a51f3bf3d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.596728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d135efa-4e89-49fa-94fb-5a51f3bf3d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.597158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d135efa-4e89-49fa-94fb-5a51f3bf3d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.597498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d135efa-4e89-49fa-94fb-5a51f3bf3d24 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.598200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d135efa-4e89-49fa-94fb-5a51f3bf3d24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.604466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d135efa-4e89-49fa-94fb-5a51f3bf3d24 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.605437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 982/1947] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:29.607183 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbb3f4-e061-4ce8-9d55-60a833944159 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.607654 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbb3f4-e061-4ce8-9d55-60a833944159 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.608085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbb3f4-e061-4ce8-9d55-60a833944159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.651846 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 966/1948] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:29 2022] POST /identity/v3/users => generated 300 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:29.652448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d135efa-4e89-49fa-94fb-5a51f3bf3d24 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.656817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5d135efa-4e89-49fa-94fb-5a51f3bf3d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.657473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-95bbb3f4-e061-4ce8-9d55-60a833944159 None None] MFA Rules not processed for user `0eba5721cc45492e9f448d77abb490a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:29.658779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-95bbb3f4-e061-4ce8-9d55-60a833944159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.686499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d135efa-4e89-49fa-94fb-5a51f3bf3d24 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.701816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.702764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.703106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.703439 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] PATH_INFO: `/v3/users/ab4a7429267a4b70a4a30c8647fd5437` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.705420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab4a7429267a4b70a4a30c8647fd5437)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.708123 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.745923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 983/1949] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:29.745923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbb3f4-e061-4ce8-9d55-60a833944159 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.745923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-95bbb3f4-e061-4ce8-9d55-60a833944159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.822053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab4a7429267a4b70a4a30c8647fd5437'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.822264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbb3f4-e061-4ce8-9d55-60a833944159 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.822775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab4a7429267a4b70a4a30c8647fd5437'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.823396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab4a7429267a4b70a4a30c8647fd5437'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.830125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6773edb-9859-46e8-a74e-2e277927a6ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.830446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6773edb-9859-46e8-a74e-2e277927a6ed service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.830780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6773edb-9859-46e8-a74e-2e277927a6ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.831057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6773edb-9859-46e8-a74e-2e277927a6ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.832939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f6773edb-9859-46e8-a74e-2e277927a6ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.849452 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6773edb-9859-46e8-a74e-2e277927a6ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.850631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6773edb-9859-46e8-a74e-2e277927a6ed service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.858931 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab4a7429267a4b70a4a30c8647fd5437'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.862019 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 984/1950] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:29.862886 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6773edb-9859-46e8-a74e-2e277927a6ed service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.864434 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f6773edb-9859-46e8-a74e-2e277927a6ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.883405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab4a7429267a4b70a4a30c8647fd5437'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:29.894727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6773edb-9859-46e8-a74e-2e277927a6ed service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.909306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca6e28-60d4-4b13-8c0e-b01235a34d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:29.910883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca6e28-60d4-4b13-8c0e-b01235a34d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:29.912455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca6e28-60d4-4b13-8c0e-b01235a34d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:29.913082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca6e28-60d4-4b13-8c0e-b01235a34d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:29.915912 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca6e28-60d4-4b13-8c0e-b01235a34d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:29.920176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca6e28-60d4-4b13-8c0e-b01235a34d08 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:29.921769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 967/1951] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:29 2022] DELETE /identity/v3/users/ab4a7429267a4b70a4a30c8647fd5437 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:29.922956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.932716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:29.950229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 985/1952] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:29.951526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca6e28-60d4-4b13-8c0e-b01235a34d08 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:29.966723 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:29.974479 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca6e28-60d4-4b13-8c0e-b01235a34d08 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:29.985922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:30.066901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf825a-fe2e-48e7-90a2-9933064ba1b1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:30.069988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f90e-74d1-48d0-9dca-b39259817331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:30.070983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6b15c-8f49-4452-8c70-308abc5e7bf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:30.071303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f90e-74d1-48d0-9dca-b39259817331 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.071601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6b15c-8f49-4452-8c70-308abc5e7bf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.071877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6b15c-8f49-4452-8c70-308abc5e7bf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.072146 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6b15c-8f49-4452-8c70-308abc5e7bf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.072388 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f90e-74d1-48d0-9dca-b39259817331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.074265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fba6b15c-8f49-4452-8c70-308abc5e7bf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.074613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f90e-74d1-48d0-9dca-b39259817331 admin admin] PATH_INFO: `/v3/projects/92f6aa90ebe84206b05ea96f46e6ca3d/users/ed2441d17a40418aac152f405063e32e/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.103199 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f90e-74d1-48d0-9dca-b39259817331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f6aa90ebe84206b05ea96f46e6ca3d, user_id=ed2441d17a40418aac152f405063e32e, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:30.108975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f90e-74d1-48d0-9dca-b39259817331 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:30.132948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 986/1953] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:29 2022] PUT /identity/v3/projects/92f6aa90ebe84206b05ea96f46e6ca3d/users/ed2441d17a40418aac152f405063e32e/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:30.137745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f90e-74d1-48d0-9dca-b39259817331 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:30.251256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6b15c-8f49-4452-8c70-308abc5e7bf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:30.254245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6b15c-8f49-4452-8c70-308abc5e7bf4 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:30.284000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 968/1954] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:30.288591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6b15c-8f49-4452-8c70-308abc5e7bf4 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:30.291432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fba6b15c-8f49-4452-8c70-308abc5e7bf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.310867 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f90e-74d1-48d0-9dca-b39259817331 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:30.322463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6b15c-8f49-4452-8c70-308abc5e7bf4 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:30.335749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1d283-e587-4975-a124-439f095f94c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:30.336205 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1d283-e587-4975-a124-439f095f94c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.336309 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:30.336494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1d283-e587-4975-a124-439f095f94c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.336819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1d283-e587-4975-a124-439f095f94c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.337401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.337521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1d283-e587-4975-a124-439f095f94c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:30.337845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.338238 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] PATH_INFO: `/v3/projects/ecc86640a4c04815b3f59912b70b01e7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.339093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1d283-e587-4975-a124-439f095f94c8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:30.342369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecc86640a4c04815b3f59912b70b01e7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:30.346580 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:30.354954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 987/1955] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:30.355782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1d283-e587-4975-a124-439f095f94c8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:30.357842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f9d1d283-e587-4975-a124-439f095f94c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.424524 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecc86640a4c04815b3f59912b70b01e7', 'request_id': 'req-7e6fa37e-04c6-446c-9781-5c2ddb04a319'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.435660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecc86640a4c04815b3f59912b70b01e7', 'request_id': 'req-7e6fa37e-04c6-446c-9781-5c2ddb04a319'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.443666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecc86640a4c04815b3f59912b70b01e7', 'request_id': 'req-7e6fa37e-04c6-446c-9781-5c2ddb04a319'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.450089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1d283-e587-4975-a124-439f095f94c8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:30.457671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] The token cache is being invalidate because project ecc86640a4c04815b3f59912b70b01e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:30.457967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.458647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.460185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 969/1956] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:30 2022] DELETE /identity/v3/projects/ecc86640a4c04815b3f59912b70b01e7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:30.460964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:30.462353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.497344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:30.507445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7674a-a018-4108-8101-ea9247be05e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:30.508238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7674a-a018-4108-8101-ea9247be05e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.508779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7674a-a018-4108-8101-ea9247be05e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.509782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7674a-a018-4108-8101-ea9247be05e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.512194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3ce7674a-a018-4108-8101-ea9247be05e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.515927 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:30.540159 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fa37e-04c6-446c-9781-5c2ddb04a319 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:30.549054 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2091cf4-dc5c-4905-b9a4-ef8682ee319d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:30.549740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2091cf4-dc5c-4905-b9a4-ef8682ee319d admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.550022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2091cf4-dc5c-4905-b9a4-ef8682ee319d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.550327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2091cf4-dc5c-4905-b9a4-ef8682ee319d admin admin] PATH_INFO: `/v3/projects/92f6aa90ebe84206b05ea96f46e6ca3d/users/ed2441d17a40418aac152f405063e32e/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.552637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2091cf4-dc5c-4905-b9a4-ef8682ee319d admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f6aa90ebe84206b05ea96f46e6ca3d, user_id=ed2441d17a40418aac152f405063e32e, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:30.555825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2091cf4-dc5c-4905-b9a4-ef8682ee319d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:30.577700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 970/1957] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:30 2022] PUT /identity/v3/projects/92f6aa90ebe84206b05ea96f46e6ca3d/users/ed2441d17a40418aac152f405063e32e/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:30.577700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4245a5b-8e9b-41e8-a41e-584c021d3dc7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.577700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4245a5b-8e9b-41e8-a41e-584c021d3dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.577700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4245a5b-8e9b-41e8-a41e-584c021d3dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.591415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d4245a5b-8e9b-41e8-a41e-584c021d3dc7 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:30.592625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d4245a5b-8e9b-41e8-a41e-584c021d3dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.597247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7674a-a018-4108-8101-ea9247be05e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:30.598680 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7674a-a018-4108-8101-ea9247be05e8 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:30.608061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 988/1958] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:30.608814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7674a-a018-4108-8101-ea9247be05e8 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:30.610294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3ce7674a-a018-4108-8101-ea9247be05e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.625174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7674a-a018-4108-8101-ea9247be05e8 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:30.656955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 971/1959] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:30.658691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92a4f1-219a-449d-be8b-e96aed1e7322 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.659036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92a4f1-219a-449d-be8b-e96aed1e7322 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.659373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92a4f1-219a-449d-be8b-e96aed1e7322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.698947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:30.700041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.700465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.700950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] PATH_INFO: `/v3/projects/88150ecec31e4f4d9a6767523a8571f2` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.702466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88150ecec31e4f4d9a6767523a8571f2)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:30.703325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-bf92a4f1-219a-449d-be8b-e96aed1e7322 None None] MFA Rules not processed for user `ed2441d17a40418aac152f405063e32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:30.704792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bf92a4f1-219a-449d-be8b-e96aed1e7322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.705486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:30.774283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88150ecec31e4f4d9a6767523a8571f2', 'request_id': 'req-7463f3a5-349a-4d5a-857c-91d6e2d64875'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.785508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88150ecec31e4f4d9a6767523a8571f2', 'request_id': 'req-7463f3a5-349a-4d5a-857c-91d6e2d64875'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.800105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 972/1960] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:30.802170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fadae-2b8d-496b-a9d7-dbf1be207eea None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.802170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fadae-2b8d-496b-a9d7-dbf1be207eea None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.802364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fadae-2b8d-496b-a9d7-dbf1be207eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.824602 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e19fadae-2b8d-496b-a9d7-dbf1be207eea None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:30.825994 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e19fadae-2b8d-496b-a9d7-dbf1be207eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.830194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88150ecec31e4f4d9a6767523a8571f2', 'request_id': 'req-7463f3a5-349a-4d5a-857c-91d6e2d64875'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.865352 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] The token cache is being invalidate because project 88150ecec31e4f4d9a6767523a8571f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:30.866700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.867409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:30.870262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 989/1961] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:30 2022] DELETE /identity/v3/projects/88150ecec31e4f4d9a6767523a8571f2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:30.871062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:30.875076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.930982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:30.938023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 973/1962] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:30.942064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c0b9f-a6bf-4752-9c6d-f8c96b23bee4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.942352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c0b9f-a6bf-4752-9c6d-f8c96b23bee4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.942628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c0b9f-a6bf-4752-9c6d-f8c96b23bee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.961216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:30.974765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463f3a5-349a-4d5a-857c-91d6e2d64875 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:30.980480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd09552-228a-4bc0-b556-d3ed3739107b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:30.981161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd09552-228a-4bc0-b556-d3ed3739107b service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:30.981481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd09552-228a-4bc0-b556-d3ed3739107b service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:30.981842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd09552-228a-4bc0-b556-d3ed3739107b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:30.984322 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1cd09552-228a-4bc0-b556-d3ed3739107b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:30.999806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c96c0b9f-a6bf-4752-9c6d-f8c96b23bee4 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:31.001428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c96c0b9f-a6bf-4752-9c6d-f8c96b23bee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:31.019912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 974/1963] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:31.022863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c0b9f-a6bf-4752-9c6d-f8c96b23bee4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.048842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c0b9f-a6bf-4752-9c6d-f8c96b23bee4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.055886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32ff07-c103-4a0c-af4b-e9318ab359ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.056621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32ff07-c103-4a0c-af4b-e9318ab359ca admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.056990 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32ff07-c103-4a0c-af4b-e9318ab359ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.057985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32ff07-c103-4a0c-af4b-e9318ab359ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.059558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32ff07-c103-4a0c-af4b-e9318ab359ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:31.060266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32ff07-c103-4a0c-af4b-e9318ab359ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.062758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32ff07-c103-4a0c-af4b-e9318ab359ca admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.105954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 975/1964] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:31.107407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb66a4d-1217-4940-858f-fd73b8c7bc18 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.107853 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb66a4d-1217-4940-858f-fd73b8c7bc18 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.109644 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb66a4d-1217-4940-858f-fd73b8c7bc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.125263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd09552-228a-4bc0-b556-d3ed3739107b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.128065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd09552-228a-4bc0-b556-d3ed3739107b service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.138516 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-0eb66a4d-1217-4940-858f-fd73b8c7bc18 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:31.139916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0eb66a4d-1217-4940-858f-fd73b8c7bc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:31.143918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 990/1965] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:31.145172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd09552-228a-4bc0-b556-d3ed3739107b service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.163421 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd09552-228a-4bc0-b556-d3ed3739107b service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.178200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 976/1966] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:31.182843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb66a4d-1217-4940-858f-fd73b8c7bc18 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.206431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733af788-1f72-4f65-be18-859c91e73cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.207063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733af788-1f72-4f65-be18-859c91e73cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.207347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733af788-1f72-4f65-be18-859c91e73cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.207618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733af788-1f72-4f65-be18-859c91e73cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.208243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733af788-1f72-4f65-be18-859c91e73cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.211371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb66a4d-1217-4940-858f-fd73b8c7bc18 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.215390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733af788-1f72-4f65-be18-859c91e73cf7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.220908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c46db0-e8df-40a3-a49e-cc6ce74fd9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.221915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c46db0-e8df-40a3-a49e-cc6ce74fd9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.222289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c46db0-e8df-40a3-a49e-cc6ce74fd9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.222741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c46db0-e8df-40a3-a49e-cc6ce74fd9d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.224859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c46db0-e8df-40a3-a49e-cc6ce74fd9d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:31.225362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c46db0-e8df-40a3-a49e-cc6ce74fd9d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.228406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c46db0-e8df-40a3-a49e-cc6ce74fd9d6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.256338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 977/1967] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:33:31.257665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c46db0-e8df-40a3-a49e-cc6ce74fd9d6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.269971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 991/1968] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:31 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:31.270888 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733af788-1f72-4f65-be18-859c91e73cf7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.271494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c46db0-e8df-40a3-a49e-cc6ce74fd9d6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.380854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3af0c-227a-4719-aa62-df084384dc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.381549 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3af0c-227a-4719-aa62-df084384dc90 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.381917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3af0c-227a-4719-aa62-df084384dc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.382389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3af0c-227a-4719-aa62-df084384dc90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.383125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3af0c-227a-4719-aa62-df084384dc90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.385425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3af0c-227a-4719-aa62-df084384dc90 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.425241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733af788-1f72-4f65-be18-859c91e73cf7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.431036 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b709b-fb56-4b43-b927-927f6f865621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.432296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b709b-fb56-4b43-b927-927f6f865621 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.432995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b709b-fb56-4b43-b927-927f6f865621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.433151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b709b-fb56-4b43-b927-927f6f865621 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.433926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b709b-fb56-4b43-b927-927f6f865621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.436084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b709b-fb56-4b43-b927-927f6f865621 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.458387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 978/1969] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:31 2022] POST /identity/v3/projects => generated 379 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:31.459991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3af0c-227a-4719-aa62-df084384dc90 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.462673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e4f3af0c-227a-4719-aa62-df084384dc90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:31.478181 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 992/1970] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:31 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:31.481957 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b709b-fb56-4b43-b927-927f6f865621 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.514639 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b709b-fb56-4b43-b927-927f6f865621 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.597172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:31.612484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f093ea0-fba3-425d-85fb-64e29d02ffaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.613370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f093ea0-fba3-425d-85fb-64e29d02ffaf admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.613937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f093ea0-fba3-425d-85fb-64e29d02ffaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.613937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f093ea0-fba3-425d-85fb-64e29d02ffaf admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.614643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f093ea0-fba3-425d-85fb-64e29d02ffaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.616773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f093ea0-fba3-425d-85fb-64e29d02ffaf admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.644816 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 993/1971] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:31 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:31.645717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f093ea0-fba3-425d-85fb-64e29d02ffaf admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.650736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3af0c-227a-4719-aa62-df084384dc90 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.658856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f093ea0-fba3-425d-85fb-64e29d02ffaf admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.664091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.664806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41371191-a3d3-45b0-b1ec-8753de8ea4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.665137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.665903 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41371191-a3d3-45b0-b1ec-8753de8ea4ad admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.666322 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41371191-a3d3-45b0-b1ec-8753de8ea4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.666883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41371191-a3d3-45b0-b1ec-8753de8ea4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.667676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41371191-a3d3-45b0-b1ec-8753de8ea4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.669777 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41371191-a3d3-45b0-b1ec-8753de8ea4ad admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.677726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.678176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] PATH_INFO: `/v3/users/eba485c920b44459bb9e652705a27b57` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.680490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eba485c920b44459bb9e652705a27b57)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.686840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.693784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 994/1972] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:31.697888 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41371191-a3d3-45b0-b1ec-8753de8ea4ad admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.712976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41371191-a3d3-45b0-b1ec-8753de8ea4ad admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.728350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40885081-7d7b-4172-9b24-74f1551f4eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.729541 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40885081-7d7b-4172-9b24-74f1551f4eee admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.729821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40885081-7d7b-4172-9b24-74f1551f4eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.730098 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40885081-7d7b-4172-9b24-74f1551f4eee admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.730770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40885081-7d7b-4172-9b24-74f1551f4eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.732362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40885081-7d7b-4172-9b24-74f1551f4eee admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.745713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 995/1973] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:31.747109 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40885081-7d7b-4172-9b24-74f1551f4eee admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.761068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40885081-7d7b-4172-9b24-74f1551f4eee admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.767491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debba55e-643b-4d3f-b320-b60b6ced234a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.768147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debba55e-643b-4d3f-b320-b60b6ced234a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.768479 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debba55e-643b-4d3f-b320-b60b6ced234a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.769430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debba55e-643b-4d3f-b320-b60b6ced234a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.770173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debba55e-643b-4d3f-b320-b60b6ced234a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.772714 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debba55e-643b-4d3f-b320-b60b6ced234a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.783296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 996/1974] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:31.787533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debba55e-643b-4d3f-b320-b60b6ced234a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.791288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eba485c920b44459bb9e652705a27b57'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.792371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eba485c920b44459bb9e652705a27b57'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.792586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eba485c920b44459bb9e652705a27b57'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.798779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:31.799083 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debba55e-643b-4d3f-b320-b60b6ced234a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.804390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e36111-858f-4084-8207-798c3934449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.805048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e36111-858f-4084-8207-798c3934449a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.805326 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e36111-858f-4084-8207-798c3934449a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.805598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e36111-858f-4084-8207-798c3934449a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.806301 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e36111-858f-4084-8207-798c3934449a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.808068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e36111-858f-4084-8207-798c3934449a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.816439 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eba485c920b44459bb9e652705a27b57'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.821115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 997/1975] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:31.822188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e36111-858f-4084-8207-798c3934449a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.826743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-75e36111-858f-4084-8207-798c3934449a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:31.830553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eba485c920b44459bb9e652705a27b57'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:31.842390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e36111-858f-4084-8207-798c3934449a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:31.851456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d26403-4461-4b69-8307-130d33c2227b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:31.852384 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d26403-4461-4b69-8307-130d33c2227b admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.853914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d26403-4461-4b69-8307-130d33c2227b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.854258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d26403-4461-4b69-8307-130d33c2227b admin admin] PATH_INFO: `/v3/projects/739be6dd1c7d4f89a50846a12f5a9f57/users/875991e7a8bc46fab19d66099e7d8059/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.855908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 979/1976] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:31 2022] DELETE /identity/v3/users/eba485c920b44459bb9e652705a27b57 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:31.858020 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:31.859991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:31.874085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d26403-4461-4b69-8307-130d33c2227b admin admin] RBAC: Authorizing `identity:create_grant(project_id=739be6dd1c7d4f89a50846a12f5a9f57, user_id=875991e7a8bc46fab19d66099e7d8059, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:31.879005 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d26403-4461-4b69-8307-130d33c2227b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:31.902372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 998/1977] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:31 2022] PUT /identity/v3/projects/739be6dd1c7d4f89a50846a12f5a9f57/users/875991e7a8bc46fab19d66099e7d8059/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:31.904400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea79beb-aeb3-4984-8175-a1c39480aa1e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:31.904716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea79beb-aeb3-4984-8175-a1c39480aa1e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:31.905057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea79beb-aeb3-4984-8175-a1c39480aa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:31.935245 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1ea79beb-aeb3-4984-8175-a1c39480aa1e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:31.936866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1ea79beb-aeb3-4984-8175-a1c39480aa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:31.955597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266a472-d4bb-4591-90e6-3a9ed1b353c7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.017335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 999/1978] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:32.019152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a5a00-076b-445b-8a39-0cd008e3869c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.019488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a5a00-076b-445b-8a39-0cd008e3869c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.019732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a5a00-076b-445b-8a39-0cd008e3869c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.050473 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32951e8c-4fec-4889-a318-8cb0a4e407fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.051251 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32951e8c-4fec-4889-a318-8cb0a4e407fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.051601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32951e8c-4fec-4889-a318-8cb0a4e407fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.051932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32951e8c-4fec-4889-a318-8cb0a4e407fd admin admin] PATH_INFO: `/v3/projects/0901f674c2114ac9b6eebd7cfa93ce68/users/794689d3b3b549b3906b671e14e3d72d/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.054775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c01a5a00-076b-445b-8a39-0cd008e3869c None None] MFA Rules not processed for user `875991e7a8bc46fab19d66099e7d8059`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:32.056412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c01a5a00-076b-445b-8a39-0cd008e3869c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:32.072338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32951e8c-4fec-4889-a318-8cb0a4e407fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0901f674c2114ac9b6eebd7cfa93ce68, user_id=794689d3b3b549b3906b671e14e3d72d, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.075625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32951e8c-4fec-4889-a318-8cb0a4e407fd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.095391 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 980/1979] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:31 2022] PUT /identity/v3/projects/0901f674c2114ac9b6eebd7cfa93ce68/users/794689d3b3b549b3906b671e14e3d72d/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:32.096413 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32951e8c-4fec-4889-a318-8cb0a4e407fd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.098116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-32951e8c-4fec-4889-a318-8cb0a4e407fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:32.164431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:32.169661 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1000/1980] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:32.171323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9d3b9-8d44-483f-b5e4-cc6bea48d51d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.172263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9d3b9-8d44-483f-b5e4-cc6bea48d51d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.173251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9d3b9-8d44-483f-b5e4-cc6bea48d51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.198840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32951e8c-4fec-4889-a318-8cb0a4e407fd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.202685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba83e8-f968-4f4a-9f2d-e549ce5a9726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.203555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba83e8-f968-4f4a-9f2d-e549ce5a9726 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.203798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba83e8-f968-4f4a-9f2d-e549ce5a9726 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.204088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba83e8-f968-4f4a-9f2d-e549ce5a9726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.233602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-39d9d3b9-8d44-483f-b5e4-cc6bea48d51d None None] MFA Rules not processed for user `794689d3b3b549b3906b671e14e3d72d`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:32.234935 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-39d9d3b9-8d44-483f-b5e4-cc6bea48d51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:32.238237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:32.304741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba83e8-f968-4f4a-9f2d-e549ce5a9726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.306205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba83e8-f968-4f4a-9f2d-e549ce5a9726 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.319430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 981/1981] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:32.320278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba83e8-f968-4f4a-9f2d-e549ce5a9726 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.331532 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba83e8-f968-4f4a-9f2d-e549ce5a9726 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.337116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943e836-758b-46b9-84be-8b227e48b656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.337744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943e836-758b-46b9-84be-8b227e48b656 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.338016 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943e836-758b-46b9-84be-8b227e48b656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.338286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943e836-758b-46b9-84be-8b227e48b656 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.338918 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943e836-758b-46b9-84be-8b227e48b656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.341001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943e836-758b-46b9-84be-8b227e48b656 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.373110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 982/1982] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:32 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:32.377090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943e836-758b-46b9-84be-8b227e48b656 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.405797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1001/1983] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:32.411089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9d3b9-8d44-483f-b5e4-cc6bea48d51d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.432233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9d3b9-8d44-483f-b5e4-cc6bea48d51d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.554517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943e836-758b-46b9-84be-8b227e48b656 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.570268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61e09e-e826-4ccc-b2b4-7692ca20f882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.571214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61e09e-e826-4ccc-b2b4-7692ca20f882 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.571501 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61e09e-e826-4ccc-b2b4-7692ca20f882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.571771 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61e09e-e826-4ccc-b2b4-7692ca20f882 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.572415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61e09e-e826-4ccc-b2b4-7692ca20f882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.576079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61e09e-e826-4ccc-b2b4-7692ca20f882 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.626525 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2db94-7a58-4ff4-8759-816cf1eb1deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.628373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2db94-7a58-4ff4-8759-816cf1eb1deb admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.628691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2db94-7a58-4ff4-8759-816cf1eb1deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.628960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2db94-7a58-4ff4-8759-816cf1eb1deb admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.629601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2db94-7a58-4ff4-8759-816cf1eb1deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.631590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2db94-7a58-4ff4-8759-816cf1eb1deb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.639883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 983/1984] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:32 2022] POST /identity/v3/users => generated 293 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:32.648630 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61e09e-e826-4ccc-b2b4-7692ca20f882 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.666434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61e09e-e826-4ccc-b2b4-7692ca20f882 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.674488 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6253d7-1c43-4883-bd03-3d8733c7bdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.675850 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6253d7-1c43-4883-bd03-3d8733c7bdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.676328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6253d7-1c43-4883-bd03-3d8733c7bdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.676725 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6253d7-1c43-4883-bd03-3d8733c7bdc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.677454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6253d7-1c43-4883-bd03-3d8733c7bdc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.679423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6253d7-1c43-4883-bd03-3d8733c7bdc9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.698100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 984/1985] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:32.702217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6253d7-1c43-4883-bd03-3d8733c7bdc9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.718241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6253d7-1c43-4883-bd03-3d8733c7bdc9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.719745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1002/1986] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:32 2022] POST /identity/v3/projects => generated 381 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:32.724343 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2db94-7a58-4ff4-8759-816cf1eb1deb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.886681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b3358-cdf1-46a2-ae2d-1f87e653f5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.887319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b3358-cdf1-46a2-ae2d-1f87e653f5dc admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.887602 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b3358-cdf1-46a2-ae2d-1f87e653f5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.887873 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b3358-cdf1-46a2-ae2d-1f87e653f5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.888540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b3358-cdf1-46a2-ae2d-1f87e653f5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.890257 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b3358-cdf1-46a2-ae2d-1f87e653f5dc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.901598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 985/1987] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:32.905421 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b3358-cdf1-46a2-ae2d-1f87e653f5dc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:32.910859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:32.917888 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b3358-cdf1-46a2-ae2d-1f87e653f5dc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.929787 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91939e81-19d2-41f7-b1af-8afe00b14c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.930434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91939e81-19d2-41f7-b1af-8afe00b14c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.930782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91939e81-19d2-41f7-b1af-8afe00b14c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.931121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91939e81-19d2-41f7-b1af-8afe00b14c42 admin admin] PATH_INFO: `/v3/projects/4ddd8bf0a0a54a0d96e36bec81fe4b76/users/36620895330f4d1bbecff9204c654113/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.945987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2db94-7a58-4ff4-8759-816cf1eb1deb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:32.953762 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5c303-4b41-4708-9a15-d9ba01d5497d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:32.955335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5c303-4b41-4708-9a15-d9ba01d5497d admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:32.955655 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5c303-4b41-4708-9a15-d9ba01d5497d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:32.956824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5c303-4b41-4708-9a15-d9ba01d5497d admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:32.957622 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5c303-4b41-4708-9a15-d9ba01d5497d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.962491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91939e81-19d2-41f7-b1af-8afe00b14c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ddd8bf0a0a54a0d96e36bec81fe4b76, user_id=36620895330f4d1bbecff9204c654113, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:32.965451 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91939e81-19d2-41f7-b1af-8afe00b14c42 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:32.965828 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5c303-4b41-4708-9a15-d9ba01d5497d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.010871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 986/1988] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:32 2022] PUT /identity/v3/projects/4ddd8bf0a0a54a0d96e36bec81fe4b76/users/36620895330f4d1bbecff9204c654113/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:33.022606 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b96da-2f89-4f89-9cd9-d0acb61247db None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.024426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b96da-2f89-4f89-9cd9-d0acb61247db None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.025041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b96da-2f89-4f89-9cd9-d0acb61247db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.032251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1003/1989] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:32 2022] POST /identity/v3/users => generated 297 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:33.036882 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5c303-4b41-4708-9a15-d9ba01d5497d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.048237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:33.058535 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5c303-4b41-4708-9a15-d9ba01d5497d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.070633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1a1b96da-2f89-4f89-9cd9-d0acb61247db None None] MFA Rules not processed for user `36620895330f4d1bbecff9204c654113`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:33.075524 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1a1b96da-2f89-4f89-9cd9-d0acb61247db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:33.153809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70154d55-68e5-4502-8fda-6bf329d33d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.154720 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70154d55-68e5-4502-8fda-6bf329d33d42 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.155092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70154d55-68e5-4502-8fda-6bf329d33d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.155572 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70154d55-68e5-4502-8fda-6bf329d33d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.156321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70154d55-68e5-4502-8fda-6bf329d33d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.158734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70154d55-68e5-4502-8fda-6bf329d33d42 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.178091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 987/1990] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:33.179015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1004/1991] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:33.180886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b96da-2f89-4f89-9cd9-d0acb61247db None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.183669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70154d55-68e5-4502-8fda-6bf329d33d42 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.210226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70154d55-68e5-4502-8fda-6bf329d33d42 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.214988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b96da-2f89-4f89-9cd9-d0acb61247db None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.217456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149831d0-4c82-4c04-a975-c781cf422030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.218242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149831d0-4c82-4c04-a975-c781cf422030 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.218755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149831d0-4c82-4c04-a975-c781cf422030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.219148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149831d0-4c82-4c04-a975-c781cf422030 admin admin] PATH_INFO: `/v3/projects/9382168813634264848991dba2294c74/users/d7ff820e4b2442c8a4ff239e27a0febe/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.221586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb9c49-e746-4bbb-83f3-e955377897fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.222486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb9c49-e746-4bbb-83f3-e955377897fe admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.222931 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb9c49-e746-4bbb-83f3-e955377897fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.223366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb9c49-e746-4bbb-83f3-e955377897fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.224852 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb9c49-e746-4bbb-83f3-e955377897fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.227705 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb9c49-e746-4bbb-83f3-e955377897fe admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.238807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149831d0-4c82-4c04-a975-c781cf422030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9382168813634264848991dba2294c74, user_id=d7ff820e4b2442c8a4ff239e27a0febe, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.242266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149831d0-4c82-4c04-a975-c781cf422030 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.281252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1005/1992] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:33 2022] PUT /identity/v3/projects/9382168813634264848991dba2294c74/users/d7ff820e4b2442c8a4ff239e27a0febe/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:33.287516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149831d0-4c82-4c04-a975-c781cf422030 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.294444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 988/1993] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:33 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:33.298508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb9c49-e746-4bbb-83f3-e955377897fe admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.436597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb9c49-e746-4bbb-83f3-e955377897fe admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.437131 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149831d0-4c82-4c04-a975-c781cf422030 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.444517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cef377-ca92-44fc-ba2f-75e70febeaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.444722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff996e-8112-4ec6-8465-ff9bfcc8f3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.445250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff996e-8112-4ec6-8465-ff9bfcc8f3ae admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.445535 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff996e-8112-4ec6-8465-ff9bfcc8f3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.445797 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cef377-ca92-44fc-ba2f-75e70febeaea admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.445900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff996e-8112-4ec6-8465-ff9bfcc8f3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.446233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cef377-ca92-44fc-ba2f-75e70febeaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.446457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff996e-8112-4ec6-8465-ff9bfcc8f3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.446810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cef377-ca92-44fc-ba2f-75e70febeaea admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.447620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cef377-ca92-44fc-ba2f-75e70febeaea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.448128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff996e-8112-4ec6-8465-ff9bfcc8f3ae admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.450085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cef377-ca92-44fc-ba2f-75e70febeaea admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.472523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1006/1994] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:33.474299 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff996e-8112-4ec6-8465-ff9bfcc8f3ae admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.476166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-13ff996e-8112-4ec6-8465-ff9bfcc8f3ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:33.495848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff996e-8112-4ec6-8465-ff9bfcc8f3ae admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.505383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.506039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.506359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.506632 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5 admin admin] PATH_INFO: `/v3/projects/40fecd21b678483c847e5207e1677db5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.507770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40fecd21b678483c847e5207e1677db5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.509788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.533226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 989/1995] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:33 2022] POST /identity/v3/users => generated 299 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:33.534426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cef377-ca92-44fc-ba2f-75e70febeaea admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.550595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cef377-ca92-44fc-ba2f-75e70febeaea admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.564675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80118708-0594-487a-9955-2346bc6688c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.564675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80118708-0594-487a-9955-2346bc6688c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.564675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80118708-0594-487a-9955-2346bc6688c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.564675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80118708-0594-487a-9955-2346bc6688c4 admin admin] PATH_INFO: `/v3/projects/9382168813634264848991dba2294c74/users/d7ff820e4b2442c8a4ff239e27a0febe/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.565240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80118708-0594-487a-9955-2346bc6688c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9382168813634264848991dba2294c74, user_id=d7ff820e4b2442c8a4ff239e27a0febe, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.565240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80118708-0594-487a-9955-2346bc6688c4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.576334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 990/1996] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:33 2022] PUT /identity/v3/projects/9382168813634264848991dba2294c74/users/d7ff820e4b2442c8a4ff239e27a0febe/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:33.577277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80118708-0594-487a-9955-2346bc6688c4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.599939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40fecd21b678483c847e5207e1677db5', 'request_id': 'req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:33.609256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40fecd21b678483c847e5207e1677db5', 'request_id': 'req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:33.614645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:33.619412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40fecd21b678483c847e5207e1677db5', 'request_id': 'req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:33.626492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:33.637990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5 admin admin] The token cache is being invalidate because project 40fecd21b678483c847e5207e1677db5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:33.638863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:33.640839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-0b4a7735-2ea0-4320-ac43-db6bbb7602d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:33.643011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1007/1997] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:33 2022] DELETE /identity/v3/projects/40fecd21b678483c847e5207e1677db5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:33.644285 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86949c-d274-4d52-9558-95112264dcf9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.644567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86949c-d274-4d52-9558-95112264dcf9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.644865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86949c-d274-4d52-9558-95112264dcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.664206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80118708-0594-487a-9955-2346bc6688c4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.676405 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ea86949c-d274-4d52-9558-95112264dcf9 None None] MFA Rules not processed for user `d7ff820e4b2442c8a4ff239e27a0febe`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:33.678019 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ea86949c-d274-4d52-9558-95112264dcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:33.727396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0debe-f8e7-4b65-828a-78e6383a210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.727935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0debe-f8e7-4b65-828a-78e6383a210e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.728186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0debe-f8e7-4b65-828a-78e6383a210e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.728434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0debe-f8e7-4b65-828a-78e6383a210e admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.729114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0debe-f8e7-4b65-828a-78e6383a210e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.730753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0debe-f8e7-4b65-828a-78e6383a210e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.742399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 991/1998] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:33.743255 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0debe-f8e7-4b65-828a-78e6383a210e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.746258 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-53e0debe-f8e7-4b65-828a-78e6383a210e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:33.759399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1008/1999] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:33.760238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86949c-d274-4d52-9558-95112264dcf9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.763581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0debe-f8e7-4b65-828a-78e6383a210e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.765659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ea86949c-d274-4d52-9558-95112264dcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:33.771166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.771879 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.772223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.772559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] PATH_INFO: `/v3/users/ed2441d17a40418aac152f405063e32e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.773807 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed2441d17a40418aac152f405063e32e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.777042 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.782969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86949c-d274-4d52-9558-95112264dcf9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.788521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18049f51-17e1-4124-8177-fca08feebba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.789245 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18049f51-17e1-4124-8177-fca08feebba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.789526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18049f51-17e1-4124-8177-fca08feebba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.789801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18049f51-17e1-4124-8177-fca08feebba8 admin admin] PATH_INFO: `/v3/projects/d601425d92824be3b23ed467489b9e89/users/0a97799731654f68aac1c14eaea9d0b5/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.807456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18049f51-17e1-4124-8177-fca08feebba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d601425d92824be3b23ed467489b9e89, user_id=0a97799731654f68aac1c14eaea9d0b5, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:33.810553 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18049f51-17e1-4124-8177-fca08feebba8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:33.828097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1009/2000] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:33 2022] PUT /identity/v3/projects/d601425d92824be3b23ed467489b9e89/users/0a97799731654f68aac1c14eaea9d0b5/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:33.831417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18049f51-17e1-4124-8177-fca08feebba8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.831417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-18049f51-17e1-4124-8177-fca08feebba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:33.833129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed2441d17a40418aac152f405063e32e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:33.833887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed2441d17a40418aac152f405063e32e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:33.834488 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed2441d17a40418aac152f405063e32e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:33.851784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed2441d17a40418aac152f405063e32e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:33.861308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed2441d17a40418aac152f405063e32e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:33.875578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 992/2001] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:33 2022] DELETE /identity/v3/users/ed2441d17a40418aac152f405063e32e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:33.877502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:33.882507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:33.901335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18049f51-17e1-4124-8177-fca08feebba8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:33.952755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:33.985143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0fc08-1a15-4d6a-a75a-53be74f93f42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:33.985777 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0fc08-1a15-4d6a-a75a-53be74f93f42 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:33.986044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0fc08-1a15-4d6a-a75a-53be74f93f42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:33.986320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0fc08-1a15-4d6a-a75a-53be74f93f42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:33.988118 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-71b0fc08-1a15-4d6a-a75a-53be74f93f42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:34.022571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53d1b6-d7ce-444f-9dc8-105c3b881dce admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:34.029074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:34.033906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb84a9-17b7-453e-be96-460e0bab0108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:34.034527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb84a9-17b7-453e-be96-460e0bab0108 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:34.034808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb84a9-17b7-453e-be96-460e0bab0108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:34.035081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb84a9-17b7-453e-be96-460e0bab0108 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:34.035742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb84a9-17b7-453e-be96-460e0bab0108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:34.037422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb84a9-17b7-453e-be96-460e0bab0108 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:34.059733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 993/2002] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:34.059733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb84a9-17b7-453e-be96-460e0bab0108 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:34.059733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b3eb84a9-17b7-453e-be96-460e0bab0108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:34.075268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb84a9-17b7-453e-be96-460e0bab0108 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:34.082194 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f13d8-d428-42f2-b26a-e3ac80750dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:34.083258 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f13d8-d428-42f2-b26a-e3ac80750dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:34.083837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f13d8-d428-42f2-b26a-e3ac80750dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:34.084368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f13d8-d428-42f2-b26a-e3ac80750dbf admin admin] PATH_INFO: `/v3/users/0c74d57f2e824c65a8daf20e0078cc10` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:34.085672 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f13d8-d428-42f2-b26a-e3ac80750dbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c74d57f2e824c65a8daf20e0078cc10)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:34.088855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f13d8-d428-42f2-b26a-e3ac80750dbf admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:34.096818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0fc08-1a15-4d6a-a75a-53be74f93f42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:34.098297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0fc08-1a15-4d6a-a75a-53be74f93f42 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:34.115289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1010/2003] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:34.117472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0fc08-1a15-4d6a-a75a-53be74f93f42 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:34.133832 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0fc08-1a15-4d6a-a75a-53be74f93f42 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:34.150695 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0999bf-e99a-4dd5-9855-757a85413c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:34.153970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0999bf-e99a-4dd5-9855-757a85413c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:34.154786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0999bf-e99a-4dd5-9855-757a85413c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:34.154786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0999bf-e99a-4dd5-9855-757a85413c84 admin admin] PATH_INFO: `/v3/projects/d601425d92824be3b23ed467489b9e89/users/0a97799731654f68aac1c14eaea9d0b5/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:34.159065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0999bf-e99a-4dd5-9855-757a85413c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d601425d92824be3b23ed467489b9e89, user_id=0a97799731654f68aac1c14eaea9d0b5, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:34.163976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0999bf-e99a-4dd5-9855-757a85413c84 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:34.171082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b26f13d8-d428-42f2-b26a-e3ac80750dbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c74d57f2e824c65a8daf20e0078cc10'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:34.172347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b26f13d8-d428-42f2-b26a-e3ac80750dbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c74d57f2e824c65a8daf20e0078cc10'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:34.173265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b26f13d8-d428-42f2-b26a-e3ac80750dbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c74d57f2e824c65a8daf20e0078cc10'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:34.194642 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1011/2004] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:34 2022] PUT /identity/v3/projects/d601425d92824be3b23ed467489b9e89/users/0a97799731654f68aac1c14eaea9d0b5/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:34.196180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9d987-98ec-45fb-8b56-b0679637445d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:34.196482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9d987-98ec-45fb-8b56-b0679637445d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:34.196764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9d987-98ec-45fb-8b56-b0679637445d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:34.207374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b26f13d8-d428-42f2-b26a-e3ac80750dbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c74d57f2e824c65a8daf20e0078cc10'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:34.221550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b26f13d8-d428-42f2-b26a-e3ac80750dbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c74d57f2e824c65a8daf20e0078cc10'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:34.243130 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 994/2005] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:34 2022] DELETE /identity/v3/users/0c74d57f2e824c65a8daf20e0078cc10 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:34.248097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2a8b8-3f37-4454-b261-44eb656e9f09 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:34.248384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2a8b8-3f37-4454-b261-44eb656e9f09 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:34.248715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2a8b8-3f37-4454-b261-44eb656e9f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:34.259880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f7a9d987-98ec-45fb-8b56-b0679637445d None None] MFA Rules not processed for user `0a97799731654f68aac1c14eaea9d0b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:34.261452 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f7a9d987-98ec-45fb-8b56-b0679637445d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:34.281751 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-00d2a8b8-3f37-4454-b261-44eb656e9f09 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:34.283424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-00d2a8b8-3f37-4454-b261-44eb656e9f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:34.431321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1012/2006] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:34.432317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9d987-98ec-45fb-8b56-b0679637445d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:34.445450 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f7a9d987-98ec-45fb-8b56-b0679637445d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:34.480582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 995/2007] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:34.485859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2a8b8-3f37-4454-b261-44eb656e9f09 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:34.491619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-00d2a8b8-3f37-4454-b261-44eb656e9f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:34.550773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9d987-98ec-45fb-8b56-b0679637445d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:34.557027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:34.557847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:34.558158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:34.558477 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] PATH_INFO: `/v3/projects/92f6aa90ebe84206b05ea96f46e6ca3d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:34.559943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f6aa90ebe84206b05ea96f46e6ca3d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:34.562358 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:34.579847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:34.587251 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2a8b8-3f37-4454-b261-44eb656e9f09 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:34.593528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4913bd57-a9fb-496c-9d06-724f6de677cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:34.594501 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913bd57-a9fb-496c-9d06-724f6de677cf service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:34.594962 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913bd57-a9fb-496c-9d06-724f6de677cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:34.595433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913bd57-a9fb-496c-9d06-724f6de677cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:34.597439 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4913bd57-a9fb-496c-9d06-724f6de677cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:34.629787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:34.642917 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f6aa90ebe84206b05ea96f46e6ca3d', 'request_id': 'req-894ee221-5cc4-4927-8674-b19346871d6b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:34.676242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f6aa90ebe84206b05ea96f46e6ca3d', 'request_id': 'req-894ee221-5cc4-4927-8674-b19346871d6b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:34.694497 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f6aa90ebe84206b05ea96f46e6ca3d', 'request_id': 'req-894ee221-5cc4-4927-8674-b19346871d6b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:34.721224 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4913bd57-a9fb-496c-9d06-724f6de677cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:34.722596 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] The token cache is being invalidate because project 92f6aa90ebe84206b05ea96f46e6ca3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:34.722901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:34.724005 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:34.729995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4913bd57-a9fb-496c-9d06-724f6de677cf service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:34.730497 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1013/2008] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:34 2022] DELETE /identity/v3/projects/92f6aa90ebe84206b05ea96f46e6ca3d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:34.731416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:34.733123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:34.748829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 996/2009] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:34.750457 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a78d1-7709-4c5f-bffb-1139e3d65364 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:34.750899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a78d1-7709-4c5f-bffb-1139e3d65364 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:34.751320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a78d1-7709-4c5f-bffb-1139e3d65364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:34.776370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f21a78d1-7709-4c5f-bffb-1139e3d65364 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:34.778652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f21a78d1-7709-4c5f-bffb-1139e3d65364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:34.838121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ee221-5cc4-4927-8674-b19346871d6b admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:34.843112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15a2e0-5a9f-4a4b-a4fa-03cc9559e37b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:34.844113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15a2e0-5a9f-4a4b-a4fa-03cc9559e37b service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:34.844316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15a2e0-5a9f-4a4b-a4fa-03cc9559e37b service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:34.844697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15a2e0-5a9f-4a4b-a4fa-03cc9559e37b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:34.847135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8a15a2e0-5a9f-4a4b-a4fa-03cc9559e37b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:34.856048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 997/2010] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:34.856533 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a78d1-7709-4c5f-bffb-1139e3d65364 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:34.857959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f21a78d1-7709-4c5f-bffb-1139e3d65364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:34.871538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15a2e0-5a9f-4a4b-a4fa-03cc9559e37b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:34.873451 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15a2e0-5a9f-4a4b-a4fa-03cc9559e37b service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:34.899814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1014/2011] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:34.900957 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6caff-8406-4f22-ad0a-e71cd7112826 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:34.901246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6caff-8406-4f22-ad0a-e71cd7112826 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:34.901643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6caff-8406-4f22-ad0a-e71cd7112826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:34.926561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d3e6caff-8406-4f22-ad0a-e71cd7112826 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:34.928210 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d3e6caff-8406-4f22-ad0a-e71cd7112826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:34.954693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1015/2012] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:34.954693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6caff-8406-4f22-ad0a-e71cd7112826 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:34.968610 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6caff-8406-4f22-ad0a-e71cd7112826 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:34.974113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781679f-d06c-4e93-a991-73a28a9dea42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:34.974812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781679f-d06c-4e93-a991-73a28a9dea42 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:34.975162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781679f-d06c-4e93-a991-73a28a9dea42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:34.975503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781679f-d06c-4e93-a991-73a28a9dea42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:34.976235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781679f-d06c-4e93-a991-73a28a9dea42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:34.976599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781679f-d06c-4e93-a991-73a28a9dea42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:34.978320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781679f-d06c-4e93-a991-73a28a9dea42 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:34.994515 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1016/2013] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:34.999065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781679f-d06c-4e93-a991-73a28a9dea42 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.005897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a78d1-7709-4c5f-bffb-1139e3d65364 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:35.012434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884d776-15ea-44bd-a7ef-b4a474605a2a tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7ff820e4b2442c8a4ff239e27a0febe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9382168813634264848991dba2294c74', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:35.013133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884d776-15ea-44bd-a7ef-b4a474605a2a tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.013462 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884d776-15ea-44bd-a7ef-b4a474605a2a tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.014630 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884d776-15ea-44bd-a7ef-b4a474605a2a tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.016234 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884d776-15ea-44bd-a7ef-b4a474605a2a tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:35.016860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884d776-15ea-44bd-a7ef-b4a474605a2a tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:35.018006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781679f-d06c-4e93-a991-73a28a9dea42 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:35.019924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f884d776-15ea-44bd-a7ef-b4a474605a2a tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:35.030976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bcdd5-dafb-4017-a32e-01caea24ffb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:35.032169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bcdd5-dafb-4017-a32e-01caea24ffb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.032993 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bcdd5-dafb-4017-a32e-01caea24ffb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.033258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bcdd5-dafb-4017-a32e-01caea24ffb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.034814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bcdd5-dafb-4017-a32e-01caea24ffb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:35.036830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bcdd5-dafb-4017-a32e-01caea24ffb9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:35.042203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 998/2014] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:35.043447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884d776-15ea-44bd-a7ef-b4a474605a2a tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.057066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884d776-15ea-44bd-a7ef-b4a474605a2a tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:35.063656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa151cd0-324b-48e3-972a-0a0d1dd59c9c tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7ff820e4b2442c8a4ff239e27a0febe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9382168813634264848991dba2294c74', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:35.064726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa151cd0-324b-48e3-972a-0a0d1dd59c9c tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.065050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa151cd0-324b-48e3-972a-0a0d1dd59c9c tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.065297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa151cd0-324b-48e3-972a-0a0d1dd59c9c tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.066109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa151cd0-324b-48e3-972a-0a0d1dd59c9c tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:35.068637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa151cd0-324b-48e3-972a-0a0d1dd59c9c tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:35.078385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1017/2015] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:34 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:35.080189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bcdd5-dafb-4017-a32e-01caea24ffb9 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.114865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 999/2016] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:35 2022] POST /identity/v3/projects => generated 355 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:35.116473 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa151cd0-324b-48e3-972a-0a0d1dd59c9c tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.149934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:35.180439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bcdd5-dafb-4017-a32e-01caea24ffb9 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:35.265085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:35.318998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa151cd0-324b-48e3-972a-0a0d1dd59c9c tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:35.324066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8200e9-1557-4f41-b1e5-1e7e5fcab0e0 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7ff820e4b2442c8a4ff239e27a0febe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9382168813634264848991dba2294c74', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:35.324939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8200e9-1557-4f41-b1e5-1e7e5fcab0e0 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.325274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8200e9-1557-4f41-b1e5-1e7e5fcab0e0 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.325614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8200e9-1557-4f41-b1e5-1e7e5fcab0e0 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.326333 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8200e9-1557-4f41-b1e5-1e7e5fcab0e0 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:35.326699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8200e9-1557-4f41-b1e5-1e7e5fcab0e0 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:35.328279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab506e-698a-4f22-819c-c99779f1175e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:35.328685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8200e9-1557-4f41-b1e5-1e7e5fcab0e0 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:35.328983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab506e-698a-4f22-819c-c99779f1175e admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.329267 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab506e-698a-4f22-819c-c99779f1175e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.329538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab506e-698a-4f22-819c-c99779f1175e admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.330236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab506e-698a-4f22-819c-c99779f1175e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:35.334988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab506e-698a-4f22-819c-c99779f1175e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:35.348838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1000/2017] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:35.375013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1018/2018] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:35 2022] POST /identity/v3/users => generated 283 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:35.377991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8200e9-1557-4f41-b1e5-1e7e5fcab0e0 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.393275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8200e9-1557-4f41-b1e5-1e7e5fcab0e0 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:35.399576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06964db-bd71-42af-b58e-00ce25c6534f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:35.400196 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06964db-bd71-42af-b58e-00ce25c6534f admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.400576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06964db-bd71-42af-b58e-00ce25c6534f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.400866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06964db-bd71-42af-b58e-00ce25c6534f admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.401926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab506e-698a-4f22-819c-c99779f1175e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.413028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06964db-bd71-42af-b58e-00ce25c6534f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:35.415482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06964db-bd71-42af-b58e-00ce25c6534f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:35.417916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab506e-698a-4f22-819c-c99779f1175e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:35.424534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7ff820e4b2442c8a4ff239e27a0febe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9382168813634264848991dba2294c74', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:35.424786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.425185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.425387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] PATH_INFO: `/v3/projects/acac12966f014f799ef353a40b1ef09c` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.426717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: Authorizing `identity:delete_project(project_id=acac12966f014f799ef353a40b1ef09c)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:35.429154 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:35.432963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1001/2019] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:35.438833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06964db-bd71-42af-b58e-00ce25c6534f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.455324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06964db-bd71-42af-b58e-00ce25c6534f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:35.466603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b73ab3-0666-4450-a545-f436d0ffbfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:35.468223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b73ab3-0666-4450-a545-f436d0ffbfff admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.468586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b73ab3-0666-4450-a545-f436d0ffbfff admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.469055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b73ab3-0666-4450-a545-f436d0ffbfff admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.469905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b73ab3-0666-4450-a545-f436d0ffbfff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:35.473319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b73ab3-0666-4450-a545-f436d0ffbfff admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:35.492696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1002/2020] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:35.497569 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b73ab3-0666-4450-a545-f436d0ffbfff admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.499378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-26b73ab3-0666-4450-a545-f436d0ffbfff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:35.501158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acac12966f014f799ef353a40b1ef09c', 'request_id': 'req-10f39c2d-38ce-4329-a7af-d40b39135c02'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.509455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acac12966f014f799ef353a40b1ef09c', 'request_id': 'req-10f39c2d-38ce-4329-a7af-d40b39135c02'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.513028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b73ab3-0666-4450-a545-f436d0ffbfff admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:35.516432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acac12966f014f799ef353a40b1ef09c', 'request_id': 'req-10f39c2d-38ce-4329-a7af-d40b39135c02'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.528216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] The token cache is being invalidate because project acac12966f014f799ef353a40b1ef09c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:35.528516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.529231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.530750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1019/2021] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:35 2022] DELETE /identity/v3/projects/acac12966f014f799ef353a40b1ef09c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:35.533415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.535259 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:35.577322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:35.578334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.578862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.579180 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] PATH_INFO: `/v3/projects/5cc5433a250544a1b3fbd6014605ad37` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.580404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc5433a250544a1b3fbd6014605ad37)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:35.582915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:35.605106 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f39c2d-38ce-4329-a7af-d40b39135c02 tempest-VolumeQuotaClassesTest-1262737471 tempest-VolumeQuotaClassesTest-1262737471-project-admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:35.610813 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bba2b3-9bd9-4965-b02b-9daaeb27e126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:35.611760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bba2b3-9bd9-4965-b02b-9daaeb27e126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.612023 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bba2b3-9bd9-4965-b02b-9daaeb27e126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.612329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bba2b3-9bd9-4965-b02b-9daaeb27e126 admin admin] PATH_INFO: `/v3/projects/f54b2e361ec943fda58660f223e28690/users/91ffe3d38d1144faba38dad3b9c39f8d/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.632897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bba2b3-9bd9-4965-b02b-9daaeb27e126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54b2e361ec943fda58660f223e28690, user_id=91ffe3d38d1144faba38dad3b9c39f8d, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:35.635897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bba2b3-9bd9-4965-b02b-9daaeb27e126 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:35.641786 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc5433a250544a1b3fbd6014605ad37', 'request_id': 'req-4c212f5d-ce05-4345-9ed1-6870efbbecd9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.649966 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc5433a250544a1b3fbd6014605ad37', 'request_id': 'req-4c212f5d-ce05-4345-9ed1-6870efbbecd9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.652102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1020/2022] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:35 2022] PUT /identity/v3/projects/f54b2e361ec943fda58660f223e28690/users/91ffe3d38d1144faba38dad3b9c39f8d/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:35.656823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc5433a250544a1b3fbd6014605ad37', 'request_id': 'req-4c212f5d-ce05-4345-9ed1-6870efbbecd9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.662955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c11264-c9e5-444a-afb4-16ba2f0be1d3 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.663238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c11264-c9e5-444a-afb4-16ba2f0be1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.663688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c11264-c9e5-444a-afb4-16ba2f0be1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.677061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] The token cache is being invalidate because project 5cc5433a250544a1b3fbd6014605ad37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:35.677357 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.678028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.679714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1003/2023] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:35 2022] DELETE /identity/v3/projects/5cc5433a250544a1b3fbd6014605ad37 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:35.683756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.685187 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:35.695410 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a7c11264-c9e5-444a-afb4-16ba2f0be1d3 None None] MFA Rules not processed for user `91ffe3d38d1144faba38dad3b9c39f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:35.697059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a7c11264-c9e5-444a-afb4-16ba2f0be1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:35.708808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:35.726550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:35.794730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c212f5d-ce05-4345-9ed1-6870efbbecd9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:35.802446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1021/2024] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:35.806364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c11264-c9e5-444a-afb4-16ba2f0be1d3 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.807910 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a7c11264-c9e5-444a-afb4-16ba2f0be1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:35.858715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:35.859359 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.859799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.860021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] PATH_INFO: `/v3/users/fa6ddfcf42be46e391132a60d301392c` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.861364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6ddfcf42be46e391132a60d301392c)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:35.863346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:35.890660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c11264-c9e5-444a-afb4-16ba2f0be1d3 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:35.897342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e0971-6345-4102-b498-c70bdd099034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:35.898064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e0971-6345-4102-b498-c70bdd099034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.898397 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e0971-6345-4102-b498-c70bdd099034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.898719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e0971-6345-4102-b498-c70bdd099034 admin admin] PATH_INFO: `/v3/users/794689d3b3b549b3906b671e14e3d72d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.900116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e0971-6345-4102-b498-c70bdd099034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=794689d3b3b549b3906b671e14e3d72d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:35.902337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e0971-6345-4102-b498-c70bdd099034 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:35.914553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6ddfcf42be46e391132a60d301392c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.915265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6ddfcf42be46e391132a60d301392c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.915890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6ddfcf42be46e391132a60d301392c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.932141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6ddfcf42be46e391132a60d301392c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.940633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6ddfcf42be46e391132a60d301392c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.962622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1004/2025] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:35 2022] DELETE /identity/v3/users/fa6ddfcf42be46e391132a60d301392c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:35.962622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:35.962947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:35.976176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-133e0971-6345-4102-b498-c70bdd099034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '794689d3b3b549b3906b671e14e3d72d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.976176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-133e0971-6345-4102-b498-c70bdd099034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '794689d3b3b549b3906b671e14e3d72d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.976176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-133e0971-6345-4102-b498-c70bdd099034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '794689d3b3b549b3906b671e14e3d72d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:35.993967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-133e0971-6345-4102-b498-c70bdd099034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '794689d3b3b549b3906b671e14e3d72d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:36.006482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-133e0971-6345-4102-b498-c70bdd099034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '794689d3b3b549b3906b671e14e3d72d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:36.019961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:36.030611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1022/2026] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:35 2022] DELETE /identity/v3/users/794689d3b3b549b3906b671e14e3d72d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:36.033136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85008ef-3bcf-42dd-8f85-425bbad41c29 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.033455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85008ef-3bcf-42dd-8f85-425bbad41c29 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.033731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85008ef-3bcf-42dd-8f85-425bbad41c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.048858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3689eed-fe3d-472b-a186-223a58953d23 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:36.059903 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c85008ef-3bcf-42dd-8f85-425bbad41c29 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:36.061442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c85008ef-3bcf-42dd-8f85-425bbad41c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:36.111717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:36.130065 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e76c6c-ed71-41bc-b60c-371df66d5bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:36.131247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76c6c-ed71-41bc-b60c-371df66d5bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.132157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76c6c-ed71-41bc-b60c-371df66d5bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.133570 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76c6c-ed71-41bc-b60c-371df66d5bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.134424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e76c6c-ed71-41bc-b60c-371df66d5bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:36.136388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e76c6c-ed71-41bc-b60c-371df66d5bb4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:36.158771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1023/2027] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:36.169262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85008ef-3bcf-42dd-8f85-425bbad41c29 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:36.171399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c85008ef-3bcf-42dd-8f85-425bbad41c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:36.180683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1005/2028] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:35 2022] POST /identity/v3/projects => generated 367 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:36.192707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e76c6c-ed71-41bc-b60c-371df66d5bb4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:36.192707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-81e76c6c-ed71-41bc-b60c-371df66d5bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:36.255238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85008ef-3bcf-42dd-8f85-425bbad41c29 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:36.288224 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e76c6c-ed71-41bc-b60c-371df66d5bb4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:36.295109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa3f18-3f85-46ec-9907-3f8e0bf7e7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:36.296254 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa3f18-3f85-46ec-9907-3f8e0bf7e7be admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.296891 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa3f18-3f85-46ec-9907-3f8e0bf7e7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.297423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa3f18-3f85-46ec-9907-3f8e0bf7e7be admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.298574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa3f18-3f85-46ec-9907-3f8e0bf7e7be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:36.301258 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa3f18-3f85-46ec-9907-3f8e0bf7e7be admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:36.335964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1006/2029] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:36 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:36.338208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa3f18-3f85-46ec-9907-3f8e0bf7e7be admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:36.356039 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa3f18-3f85-46ec-9907-3f8e0bf7e7be admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:36.368686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb2e51-4e01-495b-bdb5-ca4287f08915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:36.368902 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfcfbf-38a3-45bf-94c2-835cafea9201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:36.369490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb2e51-4e01-495b-bdb5-ca4287f08915 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.369614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfcfbf-38a3-45bf-94c2-835cafea9201 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.369824 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb2e51-4e01-495b-bdb5-ca4287f08915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.369934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfcfbf-38a3-45bf-94c2-835cafea9201 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.370081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb2e51-4e01-495b-bdb5-ca4287f08915 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.370170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfcfbf-38a3-45bf-94c2-835cafea9201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.370788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb2e51-4e01-495b-bdb5-ca4287f08915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:36.372627 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb2e51-4e01-495b-bdb5-ca4287f08915 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:36.394497 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1007/2030] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:36.401246 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb2e51-4e01-495b-bdb5-ca4287f08915 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:36.403061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfcfbf-38a3-45bf-94c2-835cafea9201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:36.404687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfcfbf-38a3-45bf-94c2-835cafea9201 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:36.416742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1024/2031] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:36.426058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb2e51-4e01-495b-bdb5-ca4287f08915 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:36.436070 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a81be-015d-4988-a9a4-3bdf8a05267f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:36.437223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a81be-015d-4988-a9a4-3bdf8a05267f admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.437947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a81be-015d-4988-a9a4-3bdf8a05267f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.438720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a81be-015d-4988-a9a4-3bdf8a05267f admin admin] PATH_INFO: `/v3/projects/af328a691c4648b9b92567603c98eafc/users/bdb28cd3e547451886e52d5ae66d5a4c/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.472863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a81be-015d-4988-a9a4-3bdf8a05267f admin admin] RBAC: Authorizing `identity:create_grant(project_id=af328a691c4648b9b92567603c98eafc, user_id=bdb28cd3e547451886e52d5ae66d5a4c, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:36.476949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a81be-015d-4988-a9a4-3bdf8a05267f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:36.500407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1008/2032] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:36 2022] PUT /identity/v3/projects/af328a691c4648b9b92567603c98eafc/users/bdb28cd3e547451886e52d5ae66d5a4c/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:36.504453 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfcfbf-38a3-45bf-94c2-835cafea9201 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:36.523324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfcfbf-38a3-45bf-94c2-835cafea9201 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:36.717841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a81be-015d-4988-a9a4-3bdf8a05267f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:36.720354 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b28a81be-015d-4988-a9a4-3bdf8a05267f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:36.738551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3170971-3e81-4c2b-99ef-6e1ef7b27a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:36.739292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170971-3e81-4c2b-99ef-6e1ef7b27a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.739733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170971-3e81-4c2b-99ef-6e1ef7b27a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.740091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170971-3e81-4c2b-99ef-6e1ef7b27a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.740945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3170971-3e81-4c2b-99ef-6e1ef7b27a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:36.742845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3170971-3e81-4c2b-99ef-6e1ef7b27a43 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:36.762963 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1025/2033] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:36.763978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3170971-3e81-4c2b-99ef-6e1ef7b27a43 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:36.777277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3170971-3e81-4c2b-99ef-6e1ef7b27a43 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:36.784927 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4bfa3-a469-45b1-abc5-20ea4f0ea053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:36.785900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4bfa3-a469-45b1-abc5-20ea4f0ea053 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.786093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4bfa3-a469-45b1-abc5-20ea4f0ea053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.786473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4bfa3-a469-45b1-abc5-20ea4f0ea053 admin admin] PATH_INFO: `/v3/projects/af328a691c4648b9b92567603c98eafc/users/bdb28cd3e547451886e52d5ae66d5a4c/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.789331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4bfa3-a469-45b1-abc5-20ea4f0ea053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af328a691c4648b9b92567603c98eafc, user_id=bdb28cd3e547451886e52d5ae66d5a4c, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:36.790074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:36.795605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4bfa3-a469-45b1-abc5-20ea4f0ea053 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:36.812795 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1026/2034] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:36 2022] PUT /identity/v3/projects/af328a691c4648b9b92567603c98eafc/users/bdb28cd3e547451886e52d5ae66d5a4c/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:36.824380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60302469-1943-46f4-8dba-73e9edc03ad3 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.824768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60302469-1943-46f4-8dba-73e9edc03ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.825090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60302469-1943-46f4-8dba-73e9edc03ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.846591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a81be-015d-4988-a9a4-3bdf8a05267f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:36.855893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-60302469-1943-46f4-8dba-73e9edc03ad3 None None] MFA Rules not processed for user `bdb28cd3e547451886e52d5ae66d5a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:36.857299 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-60302469-1943-46f4-8dba-73e9edc03ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:36.867407 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:36.917567 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63fdd2-ccd3-48f8-98d9-78778331ba30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:36.918084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63fdd2-ccd3-48f8-98d9-78778331ba30 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.918355 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63fdd2-ccd3-48f8-98d9-78778331ba30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.918617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63fdd2-ccd3-48f8-98d9-78778331ba30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.921290 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5f63fdd2-ccd3-48f8-98d9-78778331ba30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:36.976582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1027/2035] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:36.976582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60302469-1943-46f4-8dba-73e9edc03ad3 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:36.981529 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-60302469-1943-46f4-8dba-73e9edc03ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:37.024011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63fdd2-ccd3-48f8-98d9-78778331ba30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:37.025673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63fdd2-ccd3-48f8-98d9-78778331ba30 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:37.039716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1009/2036] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:37.040190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63fdd2-ccd3-48f8-98d9-78778331ba30 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:37.042694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5f63fdd2-ccd3-48f8-98d9-78778331ba30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:37.057110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63fdd2-ccd3-48f8-98d9-78778331ba30 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:37.063128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d538475-ef39-40be-be18-2b850617cfe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:37.065277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d538475-ef39-40be-be18-2b850617cfe2 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.065277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d538475-ef39-40be-be18-2b850617cfe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.065578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d538475-ef39-40be-be18-2b850617cfe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.069722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60302469-1943-46f4-8dba-73e9edc03ad3 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:37.075906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:37.076281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d538475-ef39-40be-be18-2b850617cfe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:37.076918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.077266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.077609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] PATH_INFO: `/v3/projects/0901f674c2114ac9b6eebd7cfa93ce68` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.077783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d538475-ef39-40be-be18-2b850617cfe2 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:37.079306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0901f674c2114ac9b6eebd7cfa93ce68)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:37.083137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:37.100512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1010/2037] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:37.146465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0901f674c2114ac9b6eebd7cfa93ce68', 'request_id': 'req-78ce08b7-351c-4a1a-9f74-bd0bb393687c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.156732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0901f674c2114ac9b6eebd7cfa93ce68', 'request_id': 'req-78ce08b7-351c-4a1a-9f74-bd0bb393687c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.164423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0901f674c2114ac9b6eebd7cfa93ce68', 'request_id': 'req-78ce08b7-351c-4a1a-9f74-bd0bb393687c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.176399 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] The token cache is being invalidate because project 0901f674c2114ac9b6eebd7cfa93ce68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:37.176721 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.177546 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.182078 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1028/2038] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:36 2022] DELETE /identity/v3/projects/0901f674c2114ac9b6eebd7cfa93ce68 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:37.183405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d538475-ef39-40be-be18-2b850617cfe2 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:37.190113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:37.192026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:37.197741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d538475-ef39-40be-be18-2b850617cfe2 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:37.264302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:37.287957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:37.288950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.289323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.289686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] PATH_INFO: `/v3/users/d7ff820e4b2442c8a4ff239e27a0febe` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.292573 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7ff820e4b2442c8a4ff239e27a0febe)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:37.294653 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:37.298181 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce08b7-351c-4a1a-9f74-bd0bb393687c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:37.303770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ad699-9564-481b-8fee-f055dca33e56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:37.304774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ad699-9564-481b-8fee-f055dca33e56 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.305027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ad699-9564-481b-8fee-f055dca33e56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.305300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ad699-9564-481b-8fee-f055dca33e56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.307370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6a5ad699-9564-481b-8fee-f055dca33e56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:37.339763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:37.371733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7ff820e4b2442c8a4ff239e27a0febe'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.373131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7ff820e4b2442c8a4ff239e27a0febe'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.374350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7ff820e4b2442c8a4ff239e27a0febe'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.395372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ad699-9564-481b-8fee-f055dca33e56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:37.395841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7ff820e4b2442c8a4ff239e27a0febe'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.397685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ad699-9564-481b-8fee-f055dca33e56 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:37.406560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7ff820e4b2442c8a4ff239e27a0febe'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.418154 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1029/2039] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:37.432168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1011/2040] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:37 2022] DELETE /identity/v3/users/d7ff820e4b2442c8a4ff239e27a0febe => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:37.804912 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ad699-9564-481b-8fee-f055dca33e56 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:37.810627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6a5ad699-9564-481b-8fee-f055dca33e56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:37.837822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ad699-9564-481b-8fee-f055dca33e56 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:37.950425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:37.952826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.953144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.953476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] PATH_INFO: `/v3/projects/9382168813634264848991dba2294c74` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.954691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9382168813634264848991dba2294c74)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:37.958975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:37.991378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:37.993411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.069530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9382168813634264848991dba2294c74', 'request_id': 'req-928b49f6-6c49-42ab-acca-eac81d2e4b70'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.084671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9382168813634264848991dba2294c74', 'request_id': 'req-928b49f6-6c49-42ab-acca-eac81d2e4b70'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.093007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9382168813634264848991dba2294c74', 'request_id': 'req-928b49f6-6c49-42ab-acca-eac81d2e4b70'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.095092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2a223-efa0-4af3-b60c-2dcf9aece290 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.105174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] The token cache is being invalidate because project 9382168813634264848991dba2294c74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:38.105544 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.106521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.111987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1030/2041] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:37 2022] DELETE /identity/v3/projects/9382168813634264848991dba2294c74 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:38.111987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.115232 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.144337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:38.166186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240f33d-1e85-42bc-b34d-1eb70dac2800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.166801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240f33d-1e85-42bc-b34d-1eb70dac2800 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.180999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240f33d-1e85-42bc-b34d-1eb70dac2800 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.181344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240f33d-1e85-42bc-b34d-1eb70dac2800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.187087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3240f33d-1e85-42bc-b34d-1eb70dac2800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.203746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b49f6-6c49-42ab-acca-eac81d2e4b70 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.216091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.216786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.217069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.217355 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] PATH_INFO: `/v3/projects/e9eb589e553e4f32858e8da366569c04` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.218659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9eb589e553e4f32858e8da366569c04)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.225187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.297293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9eb589e553e4f32858e8da366569c04', 'request_id': 'req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.307456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240f33d-1e85-42bc-b34d-1eb70dac2800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.309608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9eb589e553e4f32858e8da366569c04', 'request_id': 'req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.311481 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240f33d-1e85-42bc-b34d-1eb70dac2800 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.320782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9eb589e553e4f32858e8da366569c04', 'request_id': 'req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.325915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1012/2042] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:38.327489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea05b4-4d7a-40f6-acde-b22b0b366302 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.327775 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea05b4-4d7a-40f6-acde-b22b0b366302 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.328053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea05b4-4d7a-40f6-acde-b22b0b366302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.364533 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-83ea05b4-4d7a-40f6-acde-b22b0b366302 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:38.365805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] The token cache is being invalidate because project e9eb589e553e4f32858e8da366569c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:38.366098 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.369080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.369208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-83ea05b4-4d7a-40f6-acde-b22b0b366302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.372322 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1031/2043] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:38 2022] DELETE /identity/v3/projects/e9eb589e553e4f32858e8da366569c04 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:38.377953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.380462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.384007 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:38.514093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1013/2044] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:38.521742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c30bbe-c749-491a-ae19-0e133c10f4fe None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.522251 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c30bbe-c749-491a-ae19-0e133c10f4fe None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.522700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c30bbe-c749-491a-ae19-0e133c10f4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.528312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb5718-c317-4f8c-ba8f-0e78d2a79f3d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.534919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.535700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.536050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.536389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] PATH_INFO: `/v3/users/0eba5721cc45492e9f448d77abb490a7` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.537812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eba5721cc45492e9f448d77abb490a7)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.540594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.569912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a5c30bbe-c749-491a-ae19-0e133c10f4fe None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:38.571086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a5c30bbe-c749-491a-ae19-0e133c10f4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.579973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1014/2045] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:38.584605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c30bbe-c749-491a-ae19-0e133c10f4fe None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.594193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c30bbe-c749-491a-ae19-0e133c10f4fe None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.600550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c9387-a565-4b85-9750-1a6070f17bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.602030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c9387-a565-4b85-9750-1a6070f17bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.602287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c9387-a565-4b85-9750-1a6070f17bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.602558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c9387-a565-4b85-9750-1a6070f17bc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.603358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c9387-a565-4b85-9750-1a6070f17bc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:38.603658 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c9387-a565-4b85-9750-1a6070f17bc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.604674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eba5721cc45492e9f448d77abb490a7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.605533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eba5721cc45492e9f448d77abb490a7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.605648 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c9387-a565-4b85-9750-1a6070f17bc8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.606200 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eba5721cc45492e9f448d77abb490a7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.611152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:38.618776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1015/2046] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:38.622810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eba5721cc45492e9f448d77abb490a7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.623441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c9387-a565-4b85-9750-1a6070f17bc8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.625845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-553c9387-a565-4b85-9750-1a6070f17bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.632874 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eba5721cc45492e9f448d77abb490a7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:38.639337 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c9387-a565-4b85-9750-1a6070f17bc8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.644579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedc13a-d7d4-4702-bbb1-2bc7ffd91828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.645421 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedc13a-d7d4-4702-bbb1-2bc7ffd91828 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.645542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedc13a-d7d4-4702-bbb1-2bc7ffd91828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.645804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedc13a-d7d4-4702-bbb1-2bc7ffd91828 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.646400 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedc13a-d7d4-4702-bbb1-2bc7ffd91828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.648699 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1032/2047] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:38 2022] DELETE /identity/v3/users/0eba5721cc45492e9f448d77abb490a7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:38.648826 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedc13a-d7d4-4702-bbb1-2bc7ffd91828 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.679226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1016/2048] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:38 2022] POST /identity/v3/projects => generated 409 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:38.684377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.787866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7113ee-ec3c-42f1-b57b-c74713f55439 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.862132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e14c1e-2cd3-479c-a210-90355866acea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.863022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e14c1e-2cd3-479c-a210-90355866acea admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.863831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e14c1e-2cd3-479c-a210-90355866acea admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.863831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e14c1e-2cd3-479c-a210-90355866acea admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.864191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e14c1e-2cd3-479c-a210-90355866acea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.867095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e14c1e-2cd3-479c-a210-90355866acea admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.896337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1033/2049] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:38 2022] POST /identity/v3/users => generated 305 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:38.900371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedc13a-d7d4-4702-bbb1-2bc7ffd91828 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.917229 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedc13a-d7d4-4702-bbb1-2bc7ffd91828 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.932578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9548732-54c8-439a-8080-dd60a9555fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.933611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9548732-54c8-439a-8080-dd60a9555fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.934794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9548732-54c8-439a-8080-dd60a9555fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.935247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9548732-54c8-439a-8080-dd60a9555fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.936154 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9548732-54c8-439a-8080-dd60a9555fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.940213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9548732-54c8-439a-8080-dd60a9555fb7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.965738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1017/2050] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:38.970047 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e14c1e-2cd3-479c-a210-90355866acea admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.005474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9548732-54c8-439a-8080-dd60a9555fb7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.007186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c9548732-54c8-439a-8080-dd60a9555fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:39.009959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e14c1e-2cd3-479c-a210-90355866acea admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.019495 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886cd10-4df0-4d20-b439-14b966f411da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.020070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886cd10-4df0-4d20-b439-14b966f411da admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.020348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886cd10-4df0-4d20-b439-14b966f411da admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.020676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886cd10-4df0-4d20-b439-14b966f411da admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.021337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886cd10-4df0-4d20-b439-14b966f411da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.022946 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886cd10-4df0-4d20-b439-14b966f411da admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.027491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9548732-54c8-439a-8080-dd60a9555fb7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.033877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.035147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.035633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.035913 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c admin admin] PATH_INFO: `/v3/projects/1eefa17e033d452988d2c0d6630d53ca` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.049683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eefa17e033d452988d2c0d6630d53ca)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.049938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1034/2051] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:39.050158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.056959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886cd10-4df0-4d20-b439-14b966f411da admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.076255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886cd10-4df0-4d20-b439-14b966f411da admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.082352 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec1999-3694-4392-8430-ef4781eee372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.083030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec1999-3694-4392-8430-ef4781eee372 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.083309 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec1999-3694-4392-8430-ef4781eee372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.083574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec1999-3694-4392-8430-ef4781eee372 admin admin] PATH_INFO: `/v3/projects/b9789ee76dde42c183b100d175ac02c4/users/04e8f11d451e435595dc062e6d0e8300/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.115534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec1999-3694-4392-8430-ef4781eee372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9789ee76dde42c183b100d175ac02c4, user_id=04e8f11d451e435595dc062e6d0e8300, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.118292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec1999-3694-4392-8430-ef4781eee372 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.130890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eefa17e033d452988d2c0d6630d53ca', 'request_id': 'req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:39.141523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1035/2052] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:39 2022] PUT /identity/v3/projects/b9789ee76dde42c183b100d175ac02c4/users/04e8f11d451e435595dc062e6d0e8300/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:39.142295 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eefa17e033d452988d2c0d6630d53ca', 'request_id': 'req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:39.152339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c25c8-52f3-49eb-a8e8-75aa242fc302 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.152626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c25c8-52f3-49eb-a8e8-75aa242fc302 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.152959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c25c8-52f3-49eb-a8e8-75aa242fc302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.154316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eefa17e033d452988d2c0d6630d53ca', 'request_id': 'req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:39.170334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c admin admin] The token cache is being invalidate because project 1eefa17e033d452988d2c0d6630d53ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:39.170642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:39.171700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-74bbf462-41aa-4b6b-be5e-6fac3ca75c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:39.176822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1018/2053] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:39 2022] DELETE /identity/v3/projects/1eefa17e033d452988d2c0d6630d53ca => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:39.183674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11252d78-7dc5-4594-8af7-3336defb120b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.184299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11252d78-7dc5-4594-8af7-3336defb120b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.184616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11252d78-7dc5-4594-8af7-3336defb120b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.192943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-291c25c8-52f3-49eb-a8e8-75aa242fc302 None None] MFA Rules not processed for user `04e8f11d451e435595dc062e6d0e8300`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:39.195103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-291c25c8-52f3-49eb-a8e8-75aa242fc302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:39.217521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-11252d78-7dc5-4594-8af7-3336defb120b None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:39.220213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-11252d78-7dc5-4594-8af7-3336defb120b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:39.225005 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:39.285653 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:39.298713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1036/2054] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:39.304806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c25c8-52f3-49eb-a8e8-75aa242fc302 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.308431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-291c25c8-52f3-49eb-a8e8-75aa242fc302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:39.326767 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1019/2055] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:39.337647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3c94-6ac5-427a-b10b-f4040455bea2 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.338055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3c94-6ac5-427a-b10b-f4040455bea2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.338419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3c94-6ac5-427a-b10b-f4040455bea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.342908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c25c8-52f3-49eb-a8e8-75aa242fc302 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.348550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb0375-6565-4c8d-a19b-47b8b5306ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.349308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb0375-6565-4c8d-a19b-47b8b5306ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.349594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb0375-6565-4c8d-a19b-47b8b5306ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.349860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb0375-6565-4c8d-a19b-47b8b5306ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.350664 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb0375-6565-4c8d-a19b-47b8b5306ac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.353364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb0375-6565-4c8d-a19b-47b8b5306ac5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.379107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-718f3c94-6ac5-427a-b10b-f4040455bea2 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:39.383640 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-718f3c94-6ac5-427a-b10b-f4040455bea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:39.402369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1020/2056] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:39.408014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f3c94-6ac5-427a-b10b-f4040455bea2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.417452 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1037/2057] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:39 2022] POST /identity/v3/projects => generated 407 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:39.419387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb0375-6565-4c8d-a19b-47b8b5306ac5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.434026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f3c94-6ac5-427a-b10b-f4040455bea2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.534724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add4a96-8bad-4b1d-9024-ab1f7ce1a5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.537480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add4a96-8bad-4b1d-9024-ab1f7ce1a5be admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.537898 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add4a96-8bad-4b1d-9024-ab1f7ce1a5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.538689 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add4a96-8bad-4b1d-9024-ab1f7ce1a5be admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.541015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add4a96-8bad-4b1d-9024-ab1f7ce1a5be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:39.541440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add4a96-8bad-4b1d-9024-ab1f7ce1a5be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.544349 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add4a96-8bad-4b1d-9024-ab1f7ce1a5be admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.598839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb0375-6565-4c8d-a19b-47b8b5306ac5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.600536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1021/2058] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:39.605126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add4a96-8bad-4b1d-9024-ab1f7ce1a5be admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.619342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2c399-53c2-49c6-8def-85b25bb13892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.620346 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2c399-53c2-49c6-8def-85b25bb13892 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.622274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2c399-53c2-49c6-8def-85b25bb13892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.622501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2c399-53c2-49c6-8def-85b25bb13892 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.622707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add4a96-8bad-4b1d-9024-ab1f7ce1a5be admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.625252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2c399-53c2-49c6-8def-85b25bb13892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.631659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2c399-53c2-49c6-8def-85b25bb13892 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.637436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1880a-53e5-450d-bc79-9d1958e474fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.638048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1880a-53e5-450d-bc79-9d1958e474fa admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.638324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1880a-53e5-450d-bc79-9d1958e474fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.638590 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1880a-53e5-450d-bc79-9d1958e474fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.639219 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1880a-53e5-450d-bc79-9d1958e474fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.643055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1880a-53e5-450d-bc79-9d1958e474fa admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.675692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1038/2059] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:39 2022] POST /identity/v3/users => generated 310 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:39.680038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2c399-53c2-49c6-8def-85b25bb13892 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.699360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2c399-53c2-49c6-8def-85b25bb13892 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.707339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a6648-4cde-47ef-8abb-ad08cd68e0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.708662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a6648-4cde-47ef-8abb-ad08cd68e0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.709056 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a6648-4cde-47ef-8abb-ad08cd68e0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.709430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a6648-4cde-47ef-8abb-ad08cd68e0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.710211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a6648-4cde-47ef-8abb-ad08cd68e0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.712310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a6648-4cde-47ef-8abb-ad08cd68e0b3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.717869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1022/2060] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:39 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:39.719117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1880a-53e5-450d-bc79-9d1958e474fa admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.726877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1039/2061] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:39.731913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a6648-4cde-47ef-8abb-ad08cd68e0b3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.745311 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a6648-4cde-47ef-8abb-ad08cd68e0b3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.802836 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1880a-53e5-450d-bc79-9d1958e474fa admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.808808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3fdcc-1204-4dec-86e2-3d365b58c5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.809621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3fdcc-1204-4dec-86e2-3d365b58c5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.809938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3fdcc-1204-4dec-86e2-3d365b58c5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.810301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3fdcc-1204-4dec-86e2-3d365b58c5a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.810984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3fdcc-1204-4dec-86e2-3d365b58c5a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.813554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3fdcc-1204-4dec-86e2-3d365b58c5a7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.846069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1023/2062] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:39 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:39.850363 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3fdcc-1204-4dec-86e2-3d365b58c5a7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.864081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c0013-69c3-4610-8c34-3499b59e6c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.864803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c0013-69c3-4610-8c34-3499b59e6c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.865085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c0013-69c3-4610-8c34-3499b59e6c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.865534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c0013-69c3-4610-8c34-3499b59e6c87 admin admin] PATH_INFO: `/v3/projects/d2b193fb367746028aaebeee1b8165fe/users/902b1e2e038e466ea930a44dcf77ce67/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.869513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3fdcc-1204-4dec-86e2-3d365b58c5a7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.885671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89376c-1d6a-48a7-a1a5-e3661297ec06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.885907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c0013-69c3-4610-8c34-3499b59e6c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b193fb367746028aaebeee1b8165fe, user_id=902b1e2e038e466ea930a44dcf77ce67, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.886597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89376c-1d6a-48a7-a1a5-e3661297ec06 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.886877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89376c-1d6a-48a7-a1a5-e3661297ec06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.887122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89376c-1d6a-48a7-a1a5-e3661297ec06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.887783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89376c-1d6a-48a7-a1a5-e3661297ec06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.888729 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c0013-69c3-4610-8c34-3499b59e6c87 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.889432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89376c-1d6a-48a7-a1a5-e3661297ec06 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.900599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:39.907788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1040/2063] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:39 2022] PUT /identity/v3/projects/d2b193fb367746028aaebeee1b8165fe/users/902b1e2e038e466ea930a44dcf77ce67/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:39.910934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1024/2064] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:39.912335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c0013-69c3-4610-8c34-3499b59e6c87 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.915494 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89376c-1d6a-48a7-a1a5-e3661297ec06 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.948988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89376c-1d6a-48a7-a1a5-e3661297ec06 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.008432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:40.068261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c0013-69c3-4610-8c34-3499b59e6c87 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.074712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84cbd8-8e7d-479b-8834-b2e598a28e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.075455 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84cbd8-8e7d-479b-8834-b2e598a28e59 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.075885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84cbd8-8e7d-479b-8834-b2e598a28e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.076166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84cbd8-8e7d-479b-8834-b2e598a28e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.076853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84cbd8-8e7d-479b-8834-b2e598a28e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.079088 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84cbd8-8e7d-479b-8834-b2e598a28e59 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.086279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065776c-8846-4211-80f5-612cb5bd30bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.087770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065776c-8846-4211-80f5-612cb5bd30bd admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.088216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065776c-8846-4211-80f5-612cb5bd30bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.089718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065776c-8846-4211-80f5-612cb5bd30bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.090568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065776c-8846-4211-80f5-612cb5bd30bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.092382 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1041/2065] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:40.092889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065776c-8846-4211-80f5-612cb5bd30bd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.097388 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84cbd8-8e7d-479b-8834-b2e598a28e59 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.106420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1025/2066] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:40.111380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065776c-8846-4211-80f5-612cb5bd30bd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.113340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84cbd8-8e7d-479b-8834-b2e598a28e59 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.121528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6bae2-4b21-414c-9271-6b0d6a7d24b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.122124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065776c-8846-4211-80f5-612cb5bd30bd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.123625 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6bae2-4b21-414c-9271-6b0d6a7d24b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.124348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6bae2-4b21-414c-9271-6b0d6a7d24b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.125509 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6bae2-4b21-414c-9271-6b0d6a7d24b2 admin admin] PATH_INFO: `/v3/projects/d2b193fb367746028aaebeee1b8165fe/users/902b1e2e038e466ea930a44dcf77ce67/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.127049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa2023-566a-4496-821b-5da08793020c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.127704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa2023-566a-4496-821b-5da08793020c admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.127978 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa2023-566a-4496-821b-5da08793020c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.128244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa2023-566a-4496-821b-5da08793020c admin admin] PATH_INFO: `/v3/projects/1fb5607b33a646838b2f1d2d5ad569db/users/cfb6f9fe11454e1a80de751c3fafd129/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.130297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6bae2-4b21-414c-9271-6b0d6a7d24b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b193fb367746028aaebeee1b8165fe, user_id=902b1e2e038e466ea930a44dcf77ce67, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.136462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6bae2-4b21-414c-9271-6b0d6a7d24b2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.160739 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1042/2067] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:40 2022] PUT /identity/v3/projects/d2b193fb367746028aaebeee1b8165fe/users/902b1e2e038e466ea930a44dcf77ce67/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:40.160986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa2023-566a-4496-821b-5da08793020c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb5607b33a646838b2f1d2d5ad569db, user_id=cfb6f9fe11454e1a80de751c3fafd129, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.160986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa2023-566a-4496-821b-5da08793020c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.168123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ca09c-e3ac-45f8-b667-62e744e56711 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.168464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ca09c-e3ac-45f8-b667-62e744e56711 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.168986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ca09c-e3ac-45f8-b667-62e744e56711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.179681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1026/2068] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:40 2022] PUT /identity/v3/projects/1fb5607b33a646838b2f1d2d5ad569db/users/cfb6f9fe11454e1a80de751c3fafd129/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:40.191548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2f5f2-b9cc-4c69-ba69-659ed9ec9b07 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.192210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2f5f2-b9cc-4c69-ba69-659ed9ec9b07 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.192987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2f5f2-b9cc-4c69-ba69-659ed9ec9b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.216065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-529ca09c-e3ac-45f8-b667-62e744e56711 None None] MFA Rules not processed for user `902b1e2e038e466ea930a44dcf77ce67`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:40.217706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-529ca09c-e3ac-45f8-b667-62e744e56711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:40.246385 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-baa2f5f2-b9cc-4c69-ba69-659ed9ec9b07 None None] MFA Rules not processed for user `cfb6f9fe11454e1a80de751c3fafd129`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:40.247658 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-baa2f5f2-b9cc-4c69-ba69-659ed9ec9b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:40.335423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1043/2069] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:40.347695 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ca09c-e3ac-45f8-b667-62e744e56711 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.349122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-529ca09c-e3ac-45f8-b667-62e744e56711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:40.350940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1027/2070] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:40.351915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2f5f2-b9cc-4c69-ba69-659ed9ec9b07 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.450191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ca09c-e3ac-45f8-b667-62e744e56711 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.452060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2f5f2-b9cc-4c69-ba69-659ed9ec9b07 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.454302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f13a81-9fb4-4e8c-8f74-3cb265a1be48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.455098 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f13a81-9fb4-4e8c-8f74-3cb265a1be48 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.457702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b33ea-6a87-4356-a235-e84886db54a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.458858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b33ea-6a87-4356-a235-e84886db54a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.459273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b33ea-6a87-4356-a235-e84886db54a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.459591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b33ea-6a87-4356-a235-e84886db54a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.461037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b33ea-6a87-4356-a235-e84886db54a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.463075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f13a81-9fb4-4e8c-8f74-3cb265a1be48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.463257 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b33ea-6a87-4356-a235-e84886db54a3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.463416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f13a81-9fb4-4e8c-8f74-3cb265a1be48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.489806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f13a81-9fb4-4e8c-8f74-3cb265a1be48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.491342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f13a81-9fb4-4e8c-8f74-3cb265a1be48 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.500931 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1028/2071] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:40 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:40.506411 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1044/2072] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:40 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:40.506875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b33ea-6a87-4356-a235-e84886db54a3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.553298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:40.638432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b33ea-6a87-4356-a235-e84886db54a3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.650491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd767ee4-ef6e-40a1-98fb-f3312c535ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.652866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd767ee4-ef6e-40a1-98fb-f3312c535ddf admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.653162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd767ee4-ef6e-40a1-98fb-f3312c535ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.653409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd767ee4-ef6e-40a1-98fb-f3312c535ddf admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.654049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd767ee4-ef6e-40a1-98fb-f3312c535ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.656582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd767ee4-ef6e-40a1-98fb-f3312c535ddf admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.694054 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1029/2073] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:40 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:40.697657 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f13a81-9fb4-4e8c-8f74-3cb265a1be48 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.712868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f13a81-9fb4-4e8c-8f74-3cb265a1be48 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.721066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e0271-ba2b-4b9e-a296-0607f20e9e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.722941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e0271-ba2b-4b9e-a296-0607f20e9e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.723865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e0271-ba2b-4b9e-a296-0607f20e9e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.724794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e0271-ba2b-4b9e-a296-0607f20e9e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.726428 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e0271-ba2b-4b9e-a296-0607f20e9e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.732349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e0271-ba2b-4b9e-a296-0607f20e9e32 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.745518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:40.761134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1045/2074] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:40.765191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd767ee4-ef6e-40a1-98fb-f3312c535ddf admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.777653 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd767ee4-ef6e-40a1-98fb-f3312c535ddf admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.803348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e0271-ba2b-4b9e-a296-0607f20e9e32 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.813653 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fefc04-fee6-4350-ab92-7f6ee43fe5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.814698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fefc04-fee6-4350-ab92-7f6ee43fe5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.815035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fefc04-fee6-4350-ab92-7f6ee43fe5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.815369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fefc04-fee6-4350-ab92-7f6ee43fe5a1 admin admin] PATH_INFO: `/v3/projects/bc4b85ee8b7c4c02b830569b4c015836/users/a676c03833e9418e87cf0352f4c1edd7/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:40.847917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fefc04-fee6-4350-ab92-7f6ee43fe5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4b85ee8b7c4c02b830569b4c015836, user_id=a676c03833e9418e87cf0352f4c1edd7, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:40.855116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fefc04-fee6-4350-ab92-7f6ee43fe5a1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:40.898333 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1030/2075] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:40 2022] PUT /identity/v3/projects/bc4b85ee8b7c4c02b830569b4c015836/users/a676c03833e9418e87cf0352f4c1edd7/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:40.902500 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fefc04-fee6-4350-ab92-7f6ee43fe5a1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:40.905699 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e0271-ba2b-4b9e-a296-0607f20e9e32 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:40.997525 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5df3b-bfc8-4242-afe7-8777af19343a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:40.998373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5df3b-bfc8-4242-afe7-8777af19343a service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:40.998791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5df3b-bfc8-4242-afe7-8777af19343a service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:40.999311 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5df3b-bfc8-4242-afe7-8777af19343a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:41.014722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fefc04-fee6-4350-ab92-7f6ee43fe5a1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:41.020026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c36f1-117c-4c95-a58f-ec8ce285651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:41.020882 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c36f1-117c-4c95-a58f-ec8ce285651f admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:41.021166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c36f1-117c-4c95-a58f-ec8ce285651f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:41.021433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c36f1-117c-4c95-a58f-ec8ce285651f admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:41.022081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c36f1-117c-4c95-a58f-ec8ce285651f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:41.023831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c36f1-117c-4c95-a58f-ec8ce285651f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:41.041350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1031/2076] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:41.048367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c36f1-117c-4c95-a58f-ec8ce285651f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:41.066279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c36f1-117c-4c95-a58f-ec8ce285651f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:41.077377 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d1ac9-fc65-4fb7-8ba4-7271795f9f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:41.078150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d1ac9-fc65-4fb7-8ba4-7271795f9f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:41.078520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d1ac9-fc65-4fb7-8ba4-7271795f9f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:41.078847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d1ac9-fc65-4fb7-8ba4-7271795f9f5b admin admin] PATH_INFO: `/v3/projects/bc4b85ee8b7c4c02b830569b4c015836/users/a676c03833e9418e87cf0352f4c1edd7/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:41.080988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d1ac9-fc65-4fb7-8ba4-7271795f9f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4b85ee8b7c4c02b830569b4c015836, user_id=a676c03833e9418e87cf0352f4c1edd7, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:41.086146 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d1ac9-fc65-4fb7-8ba4-7271795f9f5b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:41.091359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5df3b-bfc8-4242-afe7-8777af19343a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:41.093011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5df3b-bfc8-4242-afe7-8777af19343a service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:41.099462 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1032/2077] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:41 2022] PUT /identity/v3/projects/bc4b85ee8b7c4c02b830569b4c015836/users/a676c03833e9418e87cf0352f4c1edd7/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:41.106376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1046/2078] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:40 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:41.112588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ff35f-6d55-4046-ac75-7a7b59b8b995 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:41.112975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ff35f-6d55-4046-ac75-7a7b59b8b995 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:41.113293 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ff35f-6d55-4046-ac75-7a7b59b8b995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:41.159821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-796ff35f-6d55-4046-ac75-7a7b59b8b995 None None] MFA Rules not processed for user `a676c03833e9418e87cf0352f4c1edd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:41.161340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-796ff35f-6d55-4046-ac75-7a7b59b8b995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:41.231048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:41.258940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1033/2079] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:41.398363 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5df3b-bfc8-4242-afe7-8777af19343a service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:41.399751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-60e5df3b-bfc8-4242-afe7-8777af19343a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:41.412504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5df3b-bfc8-4242-afe7-8777af19343a service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:41.495936 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec90972-1c52-4dba-a400-0fc7e031c27d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:41.496534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec90972-1c52-4dba-a400-0fc7e031c27d service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:41.496894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec90972-1c52-4dba-a400-0fc7e031c27d service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:41.497160 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec90972-1c52-4dba-a400-0fc7e031c27d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:41.522925 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:41.557728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec90972-1c52-4dba-a400-0fc7e031c27d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:41.559273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec90972-1c52-4dba-a400-0fc7e031c27d service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:41.570092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1047/2080] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:42.960972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ff35f-6d55-4046-ac75-7a7b59b8b995 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:42.960972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-796ff35f-6d55-4046-ac75-7a7b59b8b995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:43.041818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ff35f-6d55-4046-ac75-7a7b59b8b995 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:43.046639 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:43.047498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:43.047821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:43.048060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] PATH_INFO: `/v3/users/875991e7a8bc46fab19d66099e7d8059` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:43.049345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=875991e7a8bc46fab19d66099e7d8059)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:43.051406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:43.102937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '875991e7a8bc46fab19d66099e7d8059'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.103657 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '875991e7a8bc46fab19d66099e7d8059'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.104327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '875991e7a8bc46fab19d66099e7d8059'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.119665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '875991e7a8bc46fab19d66099e7d8059'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.128606 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '875991e7a8bc46fab19d66099e7d8059'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.142010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1034/2081] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:42 2022] DELETE /identity/v3/users/875991e7a8bc46fab19d66099e7d8059 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:43.146971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f28e2-9304-49d6-8a1b-c7c6ad14059a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:43.147447 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f28e2-9304-49d6-8a1b-c7c6ad14059a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:43.147577 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f28e2-9304-49d6-8a1b-c7c6ad14059a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:43.167801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2d7f28e2-9304-49d6-8a1b-c7c6ad14059a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:43.168987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2d7f28e2-9304-49d6-8a1b-c7c6ad14059a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:43.259694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1048/2082] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:43.286551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:43.292371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:43.373311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76754e06-016b-4cb4-bda3-3cd7418899d4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:43.377220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a14bda-a2e0-459c-9d50-fe1285f4f428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:43.377830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a14bda-a2e0-459c-9d50-fe1285f4f428 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:43.378108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a14bda-a2e0-459c-9d50-fe1285f4f428 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:43.378462 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a14bda-a2e0-459c-9d50-fe1285f4f428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:43.385666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f28e2-9304-49d6-8a1b-c7c6ad14059a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:43.387146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2d7f28e2-9304-49d6-8a1b-c7c6ad14059a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:43.389816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a14bda-a2e0-459c-9d50-fe1285f4f428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:43.391808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a14bda-a2e0-459c-9d50-fe1285f4f428 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:43.400740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f28e2-9304-49d6-8a1b-c7c6ad14059a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:43.401982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1035/2083] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:43.405409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:43.406038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:43.406692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:43.406960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] PATH_INFO: `/v3/users/91ffe3d38d1144faba38dad3b9c39f8d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:43.408117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ffe3d38d1144faba38dad3b9c39f8d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:43.410111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:43.459304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ffe3d38d1144faba38dad3b9c39f8d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.460085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ffe3d38d1144faba38dad3b9c39f8d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.460780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ffe3d38d1144faba38dad3b9c39f8d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.474112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ffe3d38d1144faba38dad3b9c39f8d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.481317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ffe3d38d1144faba38dad3b9c39f8d'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.494731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1049/2084] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:43 2022] DELETE /identity/v3/users/91ffe3d38d1144faba38dad3b9c39f8d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:43.499177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1d12e-c4b4-425e-8540-ade377b7559e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:43.499488 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1d12e-c4b4-425e-8540-ade377b7559e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:43.499990 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1d12e-c4b4-425e-8540-ade377b7559e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:43.522187 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-92a1d12e-c4b4-425e-8540-ade377b7559e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:43.523410 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-92a1d12e-c4b4-425e-8540-ade377b7559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:43.527618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:43.595266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1036/2085] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:43.607293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:43.608785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:43.652660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:43.673818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5c4ce-eddd-4d03-9d74-6ea1fdbcb36a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:43.677690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7062aec-7767-4bbb-b0e0-de91a9bed1bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:43.678331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7062aec-7767-4bbb-b0e0-de91a9bed1bd service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:43.678634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7062aec-7767-4bbb-b0e0-de91a9bed1bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:43.678978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7062aec-7767-4bbb-b0e0-de91a9bed1bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:43.690913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7062aec-7767-4bbb-b0e0-de91a9bed1bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:43.691803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1d12e-c4b4-425e-8540-ade377b7559e None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:43.692388 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7062aec-7767-4bbb-b0e0-de91a9bed1bd service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:43.693595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-92a1d12e-c4b4-425e-8540-ade377b7559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:43.705944 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1d12e-c4b4-425e-8540-ade377b7559e None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:43.707006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1050/2086] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:43.710759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:43.711335 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:43.711614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:43.711883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] PATH_INFO: `/v3/projects/739be6dd1c7d4f89a50846a12f5a9f57` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:43.713052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=739be6dd1c7d4f89a50846a12f5a9f57)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:43.715021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:43.784631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '739be6dd1c7d4f89a50846a12f5a9f57', 'request_id': 'req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.792988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '739be6dd1c7d4f89a50846a12f5a9f57', 'request_id': 'req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.800184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '739be6dd1c7d4f89a50846a12f5a9f57', 'request_id': 'req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.811276 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] The token cache is being invalidate because project 739be6dd1c7d4f89a50846a12f5a9f57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:43.811691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.813050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.814907 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1037/2087] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:43 2022] DELETE /identity/v3/projects/739be6dd1c7d4f89a50846a12f5a9f57 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:43.819352 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7062aec-7767-4bbb-b0e0-de91a9bed1bd service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:43.829344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7062aec-7767-4bbb-b0e0-de91a9bed1bd service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:43.892100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:43.892768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:43.893048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:43.893350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] PATH_INFO: `/v3/users/36620895330f4d1bbecff9204c654113` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:43.894473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=36620895330f4d1bbecff9204c654113)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:43.896420 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:43.949707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36620895330f4d1bbecff9204c654113'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.950456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36620895330f4d1bbecff9204c654113'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.951131 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36620895330f4d1bbecff9204c654113'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.966532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36620895330f4d1bbecff9204c654113'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:43.980882 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36620895330f4d1bbecff9204c654113'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.001666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.003707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.020714 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1051/2088] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:43 2022] DELETE /identity/v3/users/36620895330f4d1bbecff9204c654113 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:44.023417 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d3a6d-21b8-4d31-8e7e-b36eaebcf8e0 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.086421 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.087055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.087334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.087606 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] PATH_INFO: `/v3/projects/f54b2e361ec943fda58660f223e28690` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.088785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f54b2e361ec943fda58660f223e28690)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.090890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.131678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:44.142934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f54b2e361ec943fda58660f223e28690', 'request_id': 'req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.150842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f54b2e361ec943fda58660f223e28690', 'request_id': 'req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.157895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f54b2e361ec943fda58660f223e28690', 'request_id': 'req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.169662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] The token cache is being invalidate because project f54b2e361ec943fda58660f223e28690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:44.169869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.171116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.172985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1038/2089] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:44 2022] DELETE /identity/v3/projects/f54b2e361ec943fda58660f223e28690 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:44.176971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.178409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.192031 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c98ad-abff-4ca9-964c-dccdcdf85a7f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.252203 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.252873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.253149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.253414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] PATH_INFO: `/v3/users/bdb28cd3e547451886e52d5ae66d5a4c` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.254526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb28cd3e547451886e52d5ae66d5a4c)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.256468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.258089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:44.306116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.307769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.314224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb28cd3e547451886e52d5ae66d5a4c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.315057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb28cd3e547451886e52d5ae66d5a4c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.315768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb28cd3e547451886e52d5ae66d5a4c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.322620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5d68b-e1f5-46ff-b688-ae643e5a0a9e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.328539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.329532 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.329815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.330083 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] PATH_INFO: `/v3/projects/4ddd8bf0a0a54a0d96e36bec81fe4b76` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.331601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ddd8bf0a0a54a0d96e36bec81fe4b76)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.333545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.338018 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb28cd3e547451886e52d5ae66d5a4c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.351866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb28cd3e547451886e52d5ae66d5a4c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.380976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1052/2090] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:44 2022] DELETE /identity/v3/users/bdb28cd3e547451886e52d5ae66d5a4c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:44.394430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ddd8bf0a0a54a0d96e36bec81fe4b76', 'request_id': 'req-6b2a72f2-3caa-40a8-bb14-4ca993a70302'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.411553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ddd8bf0a0a54a0d96e36bec81fe4b76', 'request_id': 'req-6b2a72f2-3caa-40a8-bb14-4ca993a70302'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.419389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ddd8bf0a0a54a0d96e36bec81fe4b76', 'request_id': 'req-6b2a72f2-3caa-40a8-bb14-4ca993a70302'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.436287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] The token cache is being invalidate because project 4ddd8bf0a0a54a0d96e36bec81fe4b76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:44.436741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.437825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.440102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1039/2091] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:44 2022] DELETE /identity/v3/projects/4ddd8bf0a0a54a0d96e36bec81fe4b76 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:44.444290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.446188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.530884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bf2be-cdb7-472c-8ca8-360d06412c2f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.603165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2bdf77-cfd5-4e22-a838-41b3810f4992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.603803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bdf77-cfd5-4e22-a838-41b3810f4992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.604084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bdf77-cfd5-4e22-a838-41b3810f4992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.604360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bdf77-cfd5-4e22-a838-41b3810f4992 admin admin] PATH_INFO: `/v3/users/0a97799731654f68aac1c14eaea9d0b5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.605539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2bdf77-cfd5-4e22-a838-41b3810f4992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a97799731654f68aac1c14eaea9d0b5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.607468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2bdf77-cfd5-4e22-a838-41b3810f4992 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.636693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:44.658310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-df2bdf77-cfd5-4e22-a838-41b3810f4992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a97799731654f68aac1c14eaea9d0b5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.659000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-df2bdf77-cfd5-4e22-a838-41b3810f4992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a97799731654f68aac1c14eaea9d0b5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.659659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-df2bdf77-cfd5-4e22-a838-41b3810f4992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a97799731654f68aac1c14eaea9d0b5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.672282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-df2bdf77-cfd5-4e22-a838-41b3810f4992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a97799731654f68aac1c14eaea9d0b5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.679620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-df2bdf77-cfd5-4e22-a838-41b3810f4992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a97799731654f68aac1c14eaea9d0b5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.689955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.691718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.693574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1053/2092] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:44 2022] DELETE /identity/v3/users/0a97799731654f68aac1c14eaea9d0b5 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:44.706114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a72f2-3caa-40a8-bb14-4ca993a70302 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.773886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4937a8-7809-4b64-a52d-eec60559c9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.774460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4937a8-7809-4b64-a52d-eec60559c9c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.774740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4937a8-7809-4b64-a52d-eec60559c9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.775069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4937a8-7809-4b64-a52d-eec60559c9c3 admin admin] PATH_INFO: `/v3/projects/af328a691c4648b9b92567603c98eafc` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.776349 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4937a8-7809-4b64-a52d-eec60559c9c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af328a691c4648b9b92567603c98eafc)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.778361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4937a8-7809-4b64-a52d-eec60559c9c3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.828283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:44.829762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1d4937a8-7809-4b64-a52d-eec60559c9c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af328a691c4648b9b92567603c98eafc', 'request_id': 'req-1d4937a8-7809-4b64-a52d-eec60559c9c3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.839816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1d4937a8-7809-4b64-a52d-eec60559c9c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af328a691c4648b9b92567603c98eafc', 'request_id': 'req-1d4937a8-7809-4b64-a52d-eec60559c9c3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.847053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1d4937a8-7809-4b64-a52d-eec60559c9c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af328a691c4648b9b92567603c98eafc', 'request_id': 'req-1d4937a8-7809-4b64-a52d-eec60559c9c3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.864051 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1d4937a8-7809-4b64-a52d-eec60559c9c3 admin admin] The token cache is being invalidate because project af328a691c4648b9b92567603c98eafc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:44.864348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1d4937a8-7809-4b64-a52d-eec60559c9c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.865072 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1d4937a8-7809-4b64-a52d-eec60559c9c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:44.866594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1040/2093] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:44 2022] DELETE /identity/v3/projects/af328a691c4648b9b92567603c98eafc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:44.871604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2f762-d19c-480e-8439-b64e48f0f7e8 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.871890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2f762-d19c-480e-8439-b64e48f0f7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.872165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2f762-d19c-480e-8439-b64e48f0f7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.891769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e5f2f762-d19c-480e-8439-b64e48f0f7e8 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:44.893006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e5f2f762-d19c-480e-8439-b64e48f0f7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.952939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1054/2094] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:44.960927 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec05e7a-fc47-46bf-b738-64ed7cc24c35 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.961215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec05e7a-fc47-46bf-b738-64ed7cc24c35 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.961488 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec05e7a-fc47-46bf-b738-64ed7cc24c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.978060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2f762-d19c-480e-8439-b64e48f0f7e8 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.979479 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e5f2f762-d19c-480e-8439-b64e48f0f7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.992185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-fec05e7a-fc47-46bf-b738-64ed7cc24c35 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:44.993422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-fec05e7a-fc47-46bf-b738-64ed7cc24c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:44.994280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2f762-d19c-480e-8439-b64e48f0f7e8 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.999273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643329d-4a77-419e-87c8-81608dbf37e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.000164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643329d-4a77-419e-87c8-81608dbf37e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.000281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643329d-4a77-419e-87c8-81608dbf37e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.000482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643329d-4a77-419e-87c8-81608dbf37e4 admin admin] PATH_INFO: `/v3/projects/d601425d92824be3b23ed467489b9e89` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.001645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643329d-4a77-419e-87c8-81608dbf37e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d601425d92824be3b23ed467489b9e89)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.002328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1041/2095] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:45.003630 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643329d-4a77-419e-87c8-81608dbf37e4 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.006611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec05e7a-fc47-46bf-b738-64ed7cc24c35 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.017165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec05e7a-fc47-46bf-b738-64ed7cc24c35 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.023683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0dfc2-d667-4d89-a50b-b227f912304e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.024511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0dfc2-d667-4d89-a50b-b227f912304e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.024904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0dfc2-d667-4d89-a50b-b227f912304e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.025243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0dfc2-d667-4d89-a50b-b227f912304e admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.025972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0dfc2-d667-4d89-a50b-b227f912304e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:45.026339 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0dfc2-d667-4d89-a50b-b227f912304e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.028079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0dfc2-d667-4d89-a50b-b227f912304e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.044463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1042/2096] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:33:45.049867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0dfc2-d667-4d89-a50b-b227f912304e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.066867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0dfc2-d667-4d89-a50b-b227f912304e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.075184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d643329d-4a77-419e-87c8-81608dbf37e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd601425d92824be3b23ed467489b9e89', 'request_id': 'req-d643329d-4a77-419e-87c8-81608dbf37e4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:45.083928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d643329d-4a77-419e-87c8-81608dbf37e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd601425d92824be3b23ed467489b9e89', 'request_id': 'req-d643329d-4a77-419e-87c8-81608dbf37e4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:45.091063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d643329d-4a77-419e-87c8-81608dbf37e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd601425d92824be3b23ed467489b9e89', 'request_id': 'req-d643329d-4a77-419e-87c8-81608dbf37e4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:45.102960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d643329d-4a77-419e-87c8-81608dbf37e4 admin admin] The token cache is being invalidate because project d601425d92824be3b23ed467489b9e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:45.103326 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d643329d-4a77-419e-87c8-81608dbf37e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:45.104082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d643329d-4a77-419e-87c8-81608dbf37e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:45.105715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1055/2097] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:44 2022] DELETE /identity/v3/projects/d601425d92824be3b23ed467489b9e89 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:45.110564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc2a68-91df-4706-9c49-e741f738e8de None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.110848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc2a68-91df-4706-9c49-e741f738e8de None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.111123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc2a68-91df-4706-9c49-e741f738e8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.130419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1fcc2a68-91df-4706-9c49-e741f738e8de None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:45.131631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1fcc2a68-91df-4706-9c49-e741f738e8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:45.134418 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac1c5c-b200-4fa7-a651-79597eebb043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.135025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac1c5c-b200-4fa7-a651-79597eebb043 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.135302 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac1c5c-b200-4fa7-a651-79597eebb043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.135571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac1c5c-b200-4fa7-a651-79597eebb043 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.136190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac1c5c-b200-4fa7-a651-79597eebb043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.138135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac1c5c-b200-4fa7-a651-79597eebb043 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.139484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1056/2098] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:45.147538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f0c3a-dd20-4a09-987b-ba734d532b39 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.147821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f0c3a-dd20-4a09-987b-ba734d532b39 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.148098 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f0c3a-dd20-4a09-987b-ba734d532b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.170978 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1043/2099] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:45.171197 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-902f0c3a-dd20-4a09-987b-ba734d532b39 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:45.171197 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-902f0c3a-dd20-4a09-987b-ba734d532b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:45.172191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac1c5c-b200-4fa7-a651-79597eebb043 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.173822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-10ac1c5c-b200-4fa7-a651-79597eebb043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:45.187558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:45.241567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1057/2100] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:45.242748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f0c3a-dd20-4a09-987b-ba734d532b39 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.252174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f0c3a-dd20-4a09-987b-ba734d532b39 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.255105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac1c5c-b200-4fa7-a651-79597eebb043 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.256915 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ae26b-1fe2-486f-ab50-1c8209385a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.257519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ae26b-1fe2-486f-ab50-1c8209385a80 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.257800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ae26b-1fe2-486f-ab50-1c8209385a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.258070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ae26b-1fe2-486f-ab50-1c8209385a80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.258785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ae26b-1fe2-486f-ab50-1c8209385a80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:45.259091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ae26b-1fe2-486f-ab50-1c8209385a80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.260607 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8e9f4-889e-44a5-9fca-f442d4c2f320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.260738 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ae26b-1fe2-486f-ab50-1c8209385a80 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.261890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8e9f4-889e-44a5-9fca-f442d4c2f320 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.262176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8e9f4-889e-44a5-9fca-f442d4c2f320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.262443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8e9f4-889e-44a5-9fca-f442d4c2f320 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.263105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8e9f4-889e-44a5-9fca-f442d4c2f320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.265155 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8e9f4-889e-44a5-9fca-f442d4c2f320 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.272722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1058/2101] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:45.277122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ae26b-1fe2-486f-ab50-1c8209385a80 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.286884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ae26b-1fe2-486f-ab50-1c8209385a80 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.287511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1044/2102] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:45.291588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8e9f4-889e-44a5-9fca-f442d4c2f320 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.292900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40c8c6-dc36-4229-9a37-2f149a0c4e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.293496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40c8c6-dc36-4229-9a37-2f149a0c4e6a admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.293775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40c8c6-dc36-4229-9a37-2f149a0c4e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.294044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40c8c6-dc36-4229-9a37-2f149a0c4e6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.294664 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40c8c6-dc36-4229-9a37-2f149a0c4e6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.296561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40c8c6-dc36-4229-9a37-2f149a0c4e6a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.302175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8e9f4-889e-44a5-9fca-f442d4c2f320 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.306917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef02d2b-2020-486c-a0f0-204c6aefe5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.307566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef02d2b-2020-486c-a0f0-204c6aefe5fd admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.308012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef02d2b-2020-486c-a0f0-204c6aefe5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.308278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef02d2b-2020-486c-a0f0-204c6aefe5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.308962 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef02d2b-2020-486c-a0f0-204c6aefe5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.310664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef02d2b-2020-486c-a0f0-204c6aefe5fd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.322176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1045/2103] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:45.326150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef02d2b-2020-486c-a0f0-204c6aefe5fd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.330420 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1059/2104] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:45.331330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40c8c6-dc36-4229-9a37-2f149a0c4e6a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.337103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef02d2b-2020-486c-a0f0-204c6aefe5fd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.355894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:45.399851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880bb87-ef04-45d7-b251-94f9e52bcd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.400264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40c8c6-dc36-4229-9a37-2f149a0c4e6a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.400453 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880bb87-ef04-45d7-b251-94f9e52bcd98 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.400754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880bb87-ef04-45d7-b251-94f9e52bcd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.401021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880bb87-ef04-45d7-b251-94f9e52bcd98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.401739 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d880bb87-ef04-45d7-b251-94f9e52bcd98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.403292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d880bb87-ef04-45d7-b251-94f9e52bcd98 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.405202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d70a8-f58e-461d-bf69-711469a33412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.406226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d70a8-f58e-461d-bf69-711469a33412 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.406512 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d70a8-f58e-461d-bf69-711469a33412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.406919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d70a8-f58e-461d-bf69-711469a33412 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.407558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d70a8-f58e-461d-bf69-711469a33412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.409967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d70a8-f58e-461d-bf69-711469a33412 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.413982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1046/2105] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:45.418160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880bb87-ef04-45d7-b251-94f9e52bcd98 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.430126 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880bb87-ef04-45d7-b251-94f9e52bcd98 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.435313 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ca616-86f4-4a44-9d26-9f849bb52c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.435984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ca616-86f4-4a44-9d26-9f849bb52c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.436296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ca616-86f4-4a44-9d26-9f849bb52c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.436600 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ca616-86f4-4a44-9d26-9f849bb52c66 admin admin] PATH_INFO: `/v3/projects/e9c723bfdfbf4bc7b25dcf38aeca7dcd/users/db03aee658964aad804841b4a4191be8/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.443052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1060/2106] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/users => generated 282 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:45.447555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d70a8-f58e-461d-bf69-711469a33412 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.452557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ca616-86f4-4a44-9d26-9f849bb52c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c723bfdfbf4bc7b25dcf38aeca7dcd, user_id=db03aee658964aad804841b4a4191be8, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.455378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ca616-86f4-4a44-9d26-9f849bb52c66 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.458710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d70a8-f58e-461d-bf69-711469a33412 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.463859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ca76a-b9ca-4dee-acf4-8f9afc231c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.464457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ca76a-b9ca-4dee-acf4-8f9afc231c42 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.464763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ca76a-b9ca-4dee-acf4-8f9afc231c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.465035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ca76a-b9ca-4dee-acf4-8f9afc231c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.465682 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ca76a-b9ca-4dee-acf4-8f9afc231c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.467274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ca76a-b9ca-4dee-acf4-8f9afc231c42 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.468832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1047/2107] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:45 2022] PUT /identity/v3/projects/e9c723bfdfbf4bc7b25dcf38aeca7dcd/users/db03aee658964aad804841b4a4191be8/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:45.478309 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1061/2108] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:45.479388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17549596-299e-4ac0-94b0-f6c2e90af6c5 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.479668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17549596-299e-4ac0-94b0-f6c2e90af6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.479933 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17549596-299e-4ac0-94b0-f6c2e90af6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.481983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ca76a-b9ca-4dee-acf4-8f9afc231c42 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.492704 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ca76a-b9ca-4dee-acf4-8f9afc231c42 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.508280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-17549596-299e-4ac0-94b0-f6c2e90af6c5 None None] MFA Rules not processed for user `db03aee658964aad804841b4a4191be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:45.509865 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-17549596-299e-4ac0-94b0-f6c2e90af6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:45.550498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367c4b2-e432-4058-a8a9-37174b9e2199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.551017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367c4b2-e432-4058-a8a9-37174b9e2199 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.551291 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367c4b2-e432-4058-a8a9-37174b9e2199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.551562 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367c4b2-e432-4058-a8a9-37174b9e2199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.552226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367c4b2-e432-4058-a8a9-37174b9e2199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.553854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367c4b2-e432-4058-a8a9-37174b9e2199 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.564923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1062/2109] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:45.568845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367c4b2-e432-4058-a8a9-37174b9e2199 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.574707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:45.575646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1048/2110] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:45.580365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17549596-299e-4ac0-94b0-f6c2e90af6c5 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.580796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367c4b2-e432-4058-a8a9-37174b9e2199 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.589115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f393d-1af1-4035-831b-9f2af1ef2b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.589905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f393d-1af1-4035-831b-9f2af1ef2b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.590326 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f393d-1af1-4035-831b-9f2af1ef2b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.591396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17549596-299e-4ac0-94b0-f6c2e90af6c5 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.596538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d060d58-a382-4db6-a22e-a103bac173f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.597167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d060d58-a382-4db6-a22e-a103bac173f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.597441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d060d58-a382-4db6-a22e-a103bac173f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.597719 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d060d58-a382-4db6-a22e-a103bac173f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.598341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d060d58-a382-4db6-a22e-a103bac173f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.599562 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f393d-1af1-4035-831b-9f2af1ef2b6e admin admin] PATH_INFO: `/v3/projects/ad98afd2f9c444fe96b8577bfcdb065f/users/d76439e4a1624e508566094cfde5c64e/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.600268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d060d58-a382-4db6-a22e-a103bac173f9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.616624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f393d-1af1-4035-831b-9f2af1ef2b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad98afd2f9c444fe96b8577bfcdb065f, user_id=d76439e4a1624e508566094cfde5c64e, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.619442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f393d-1af1-4035-831b-9f2af1ef2b6e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.632142 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1049/2111] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:45.636172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1063/2112] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:45 2022] PUT /identity/v3/projects/ad98afd2f9c444fe96b8577bfcdb065f/users/d76439e4a1624e508566094cfde5c64e/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:45.637100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d060d58-a382-4db6-a22e-a103bac173f9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.647619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea03ccb-1667-477c-8fd6-b5de3a8d0d0b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.647874 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea03ccb-1667-477c-8fd6-b5de3a8d0d0b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.648180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea03ccb-1667-477c-8fd6-b5de3a8d0d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.679226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-bea03ccb-1667-477c-8fd6-b5de3a8d0d0b None None] MFA Rules not processed for user `d76439e4a1624e508566094cfde5c64e`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:45.680576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bea03ccb-1667-477c-8fd6-b5de3a8d0d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:45.722780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d060d58-a382-4db6-a22e-a103bac173f9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.728383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226d5ef-b248-4708-8d5a-f9d0d354e74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.729208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226d5ef-b248-4708-8d5a-f9d0d354e74d admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.729634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226d5ef-b248-4708-8d5a-f9d0d354e74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.730121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226d5ef-b248-4708-8d5a-f9d0d354e74d admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.730762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226d5ef-b248-4708-8d5a-f9d0d354e74d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.732763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226d5ef-b248-4708-8d5a-f9d0d354e74d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.741020 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:45.755435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1064/2113] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:45.757534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1050/2114] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:45.761265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea03ccb-1667-477c-8fd6-b5de3a8d0d0b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.770675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226d5ef-b248-4708-8d5a-f9d0d354e74d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.772534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7226d5ef-b248-4708-8d5a-f9d0d354e74d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:45.772731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea03ccb-1667-477c-8fd6-b5de3a8d0d0b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.777273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e55071-2786-4b73-b47f-ab58acbf444a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.777883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e55071-2786-4b73-b47f-ab58acbf444a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.778162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e55071-2786-4b73-b47f-ab58acbf444a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.778431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e55071-2786-4b73-b47f-ab58acbf444a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.779079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e55071-2786-4b73-b47f-ab58acbf444a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.780706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e55071-2786-4b73-b47f-ab58acbf444a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.791248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1065/2115] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:45.795278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e55071-2786-4b73-b47f-ab58acbf444a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.805158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e55071-2786-4b73-b47f-ab58acbf444a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.811274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e18ed8-a3f1-4c0a-8a58-f45a8b26d383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.811960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e18ed8-a3f1-4c0a-8a58-f45a8b26d383 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.812194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e18ed8-a3f1-4c0a-8a58-f45a8b26d383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.812498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e18ed8-a3f1-4c0a-8a58-f45a8b26d383 admin admin] PATH_INFO: `/v3/projects/29510dc53bbb4201964ea85666d9e30b/users/d708ce334e934439bbab51c7fedc5965/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.829049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e18ed8-a3f1-4c0a-8a58-f45a8b26d383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29510dc53bbb4201964ea85666d9e30b, user_id=d708ce334e934439bbab51c7fedc5965, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.831563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226d5ef-b248-4708-8d5a-f9d0d354e74d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.831705 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e18ed8-a3f1-4c0a-8a58-f45a8b26d383 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.835517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d805a-845f-44a6-aff8-29d700b6ec7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.836242 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d805a-845f-44a6-aff8-29d700b6ec7c service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.836621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d805a-845f-44a6-aff8-29d700b6ec7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.837077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d805a-845f-44a6-aff8-29d700b6ec7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.846780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d805a-845f-44a6-aff8-29d700b6ec7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.848259 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d805a-845f-44a6-aff8-29d700b6ec7c service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.850615 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1066/2116] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:45 2022] PUT /identity/v3/projects/29510dc53bbb4201964ea85666d9e30b/users/d708ce334e934439bbab51c7fedc5965/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:45.854566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e18ed8-a3f1-4c0a-8a58-f45a8b26d383 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.858329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1051/2117] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:45.916377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e18ed8-a3f1-4c0a-8a58-f45a8b26d383 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.921973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22220413-875a-4462-879e-c42010623a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.922670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22220413-875a-4462-879e-c42010623a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.922961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22220413-875a-4462-879e-c42010623a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.923263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22220413-875a-4462-879e-c42010623a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.923901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22220413-875a-4462-879e-c42010623a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.925502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22220413-875a-4462-879e-c42010623a9a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.936877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1067/2118] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:45.941669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d805a-845f-44a6-aff8-29d700b6ec7c service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.952759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d805a-845f-44a6-aff8-29d700b6ec7c service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.957801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db0c14-1aa6-4eb8-9fd5-282ea353714f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.958456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db0c14-1aa6-4eb8-9fd5-282ea353714f admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.958733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db0c14-1aa6-4eb8-9fd5-282ea353714f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.959005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db0c14-1aa6-4eb8-9fd5-282ea353714f admin admin] PATH_INFO: `/v3/projects/29510dc53bbb4201964ea85666d9e30b/users/d708ce334e934439bbab51c7fedc5965/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.961752 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db0c14-1aa6-4eb8-9fd5-282ea353714f admin admin] RBAC: Authorizing `identity:create_grant(project_id=29510dc53bbb4201964ea85666d9e30b, user_id=d708ce334e934439bbab51c7fedc5965, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.964563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db0c14-1aa6-4eb8-9fd5-282ea353714f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.975360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1052/2119] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:45 2022] PUT /identity/v3/projects/29510dc53bbb4201964ea85666d9e30b/users/d708ce334e934439bbab51c7fedc5965/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:45.992628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f40abc-e0ee-4920-ba9d-900a5da98dd0 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.992964 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f40abc-e0ee-4920-ba9d-900a5da98dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.993259 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f40abc-e0ee-4920-ba9d-900a5da98dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:46.022032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-47f40abc-e0ee-4920-ba9d-900a5da98dd0 None None] MFA Rules not processed for user `d708ce334e934439bbab51c7fedc5965`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:46.023285 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-47f40abc-e0ee-4920-ba9d-900a5da98dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:46.041308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:46.104137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1068/2120] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:46.117021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db0c14-1aa6-4eb8-9fd5-282ea353714f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:46.119050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-47db0c14-1aa6-4eb8-9fd5-282ea353714f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:46.186889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db0c14-1aa6-4eb8-9fd5-282ea353714f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:46.191876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cdb41-8d56-4131-875f-cb30b2d066d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:46.192633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cdb41-8d56-4131-875f-cb30b2d066d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:46.193018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cdb41-8d56-4131-875f-cb30b2d066d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:46.193345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cdb41-8d56-4131-875f-cb30b2d066d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:46.205463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11cdb41-8d56-4131-875f-cb30b2d066d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:46.206950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11cdb41-8d56-4131-875f-cb30b2d066d3 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:46.217938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1053/2121] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:46.840169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f40abc-e0ee-4920-ba9d-900a5da98dd0 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:46.914953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f40abc-e0ee-4920-ba9d-900a5da98dd0 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:46.920812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:46.921857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:46.921857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:46.922701 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] PATH_INFO: `/v3/users/db03aee658964aad804841b4a4191be8` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:46.923529 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db03aee658964aad804841b4a4191be8)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:46.925846 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:46.987877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db03aee658964aad804841b4a4191be8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:46.989696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db03aee658964aad804841b4a4191be8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:46.990313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db03aee658964aad804841b4a4191be8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.003170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db03aee658964aad804841b4a4191be8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.010798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db03aee658964aad804841b4a4191be8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.023588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1069/2122] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:46 2022] DELETE /identity/v3/users/db03aee658964aad804841b4a4191be8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:47.028229 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a7e64-3bc8-4c7a-bae1-c6e387d9d795 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:47.028518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a7e64-3bc8-4c7a-bae1-c6e387d9d795 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:47.028822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a7e64-3bc8-4c7a-bae1-c6e387d9d795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:47.048328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-9d4a7e64-3bc8-4c7a-bae1-c6e387d9d795 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:47.049565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9d4a7e64-3bc8-4c7a-bae1-c6e387d9d795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:47.111792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1054/2123] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:47.121821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:47.123233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:47.165240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:47.194086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d05e28-918d-4d30-9629-08ea104fb2f0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:47.198588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9794345-3688-46bf-a2cd-d336d6e9fedb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:47.198588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9794345-3688-46bf-a2cd-d336d6e9fedb service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:47.198945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9794345-3688-46bf-a2cd-d336d6e9fedb service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:47.199220 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9794345-3688-46bf-a2cd-d336d6e9fedb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:47.209921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9794345-3688-46bf-a2cd-d336d6e9fedb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:47.211433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9794345-3688-46bf-a2cd-d336d6e9fedb service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:47.222928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1070/2124] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:47.526450 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a7e64-3bc8-4c7a-bae1-c6e387d9d795 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:47.528158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-9d4a7e64-3bc8-4c7a-bae1-c6e387d9d795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:47.538347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9794345-3688-46bf-a2cd-d336d6e9fedb service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:47.539830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a9794345-3688-46bf-a2cd-d336d6e9fedb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:47.542676 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a7e64-3bc8-4c7a-bae1-c6e387d9d795 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:47.559624 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba46d04-ea1a-4d80-8a3c-af7a6d64065f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:47.560373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba46d04-ea1a-4d80-8a3c-af7a6d64065f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:47.560941 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba46d04-ea1a-4d80-8a3c-af7a6d64065f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:47.561231 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba46d04-ea1a-4d80-8a3c-af7a6d64065f admin admin] PATH_INFO: `/v3/users/cfb6f9fe11454e1a80de751c3fafd129` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:47.562902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba46d04-ea1a-4d80-8a3c-af7a6d64065f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfb6f9fe11454e1a80de751c3fafd129)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:47.565976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba46d04-ea1a-4d80-8a3c-af7a6d64065f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:47.568491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9794345-3688-46bf-a2cd-d336d6e9fedb service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:47.575987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:47.575987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:47.576199 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:47.576384 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] PATH_INFO: `/v3/projects/e9c723bfdfbf4bc7b25dcf38aeca7dcd` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:47.579179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9c723bfdfbf4bc7b25dcf38aeca7dcd)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:47.581209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:47.584162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:INSERT=4,keystone:SELECT=95,keystone:DELETE=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:47.621586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0ba46d04-ea1a-4d80-8a3c-af7a6d64065f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfb6f9fe11454e1a80de751c3fafd129'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.622323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0ba46d04-ea1a-4d80-8a3c-af7a6d64065f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfb6f9fe11454e1a80de751c3fafd129'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.623067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0ba46d04-ea1a-4d80-8a3c-af7a6d64065f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfb6f9fe11454e1a80de751c3fafd129'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.636676 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0ba46d04-ea1a-4d80-8a3c-af7a6d64065f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfb6f9fe11454e1a80de751c3fafd129'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.647288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9c723bfdfbf4bc7b25dcf38aeca7dcd', 'request_id': 'req-6b65bb45-39f1-4362-97e1-04a41cd344e7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.648491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-0ba46d04-ea1a-4d80-8a3c-af7a6d64065f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfb6f9fe11454e1a80de751c3fafd129'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.663680 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9c723bfdfbf4bc7b25dcf38aeca7dcd', 'request_id': 'req-6b65bb45-39f1-4362-97e1-04a41cd344e7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.671295 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1055/2125] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:47 2022] DELETE /identity/v3/users/cfb6f9fe11454e1a80de751c3fafd129 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:47.674612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9c723bfdfbf4bc7b25dcf38aeca7dcd', 'request_id': 'req-6b65bb45-39f1-4362-97e1-04a41cd344e7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.678512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cab277-b2eb-48fa-8544-87fceaf032d7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:47.678874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cab277-b2eb-48fa-8544-87fceaf032d7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:47.679204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cab277-b2eb-48fa-8544-87fceaf032d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:47.694782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] The token cache is being invalidate because project e9c723bfdfbf4bc7b25dcf38aeca7dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:47.695300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.696313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.698672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1071/2126] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:47 2022] DELETE /identity/v3/projects/e9c723bfdfbf4bc7b25dcf38aeca7dcd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:47.704583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:47.706659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:47.708228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-97cab277-b2eb-48fa-8544-87fceaf032d7 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:47.709644 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-97cab277-b2eb-48fa-8544-87fceaf032d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:47.794552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1056/2127] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:47.810325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cab277-b2eb-48fa-8544-87fceaf032d7 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:47.814072 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-97cab277-b2eb-48fa-8544-87fceaf032d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:47.822536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65bb45-39f1-4362-97e1-04a41cd344e7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:47.830524 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:47.831648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:47.831980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:47.832306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] PATH_INFO: `/v3/users/d708ce334e934439bbab51c7fedc5965` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:47.833616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d708ce334e934439bbab51c7fedc5965)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:47.835680 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:47.894697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd708ce334e934439bbab51c7fedc5965'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.895837 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd708ce334e934439bbab51c7fedc5965'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.896446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd708ce334e934439bbab51c7fedc5965'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.910758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:47.914000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd708ce334e934439bbab51c7fedc5965'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.926318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd708ce334e934439bbab51c7fedc5965'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.933066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cab277-b2eb-48fa-8544-87fceaf032d7 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:47.937596 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e638f-7b21-45f2-91d7-89c167afd045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:47.938796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e638f-7b21-45f2-91d7-89c167afd045 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:47.939045 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e638f-7b21-45f2-91d7-89c167afd045 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:47.939314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e638f-7b21-45f2-91d7-89c167afd045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:47.941543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-722e638f-7b21-45f2-91d7-89c167afd045 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:47.945306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1072/2128] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:47 2022] DELETE /identity/v3/users/d708ce334e934439bbab51c7fedc5965 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:47.954828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e638f-7b21-45f2-91d7-89c167afd045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:47.956276 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e638f-7b21-45f2-91d7-89c167afd045 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:47.967145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1057/2129] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:48.437055 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.439822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:48.509454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e638f-7b21-45f2-91d7-89c167afd045 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.511330 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-722e638f-7b21-45f2-91d7-89c167afd045 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:48.524958 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e638f-7b21-45f2-91d7-89c167afd045 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.544048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4ef28-bb30-4858-b3a5-ce437010edcb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.553080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfb150-8eb2-40db-b72f-a1f8fc5af63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.553724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfb150-8eb2-40db-b72f-a1f8fc5af63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.554004 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfb150-8eb2-40db-b72f-a1f8fc5af63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.554275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfb150-8eb2-40db-b72f-a1f8fc5af63f admin admin] PATH_INFO: `/v3/projects/29510dc53bbb4201964ea85666d9e30b` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.555414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfb150-8eb2-40db-b72f-a1f8fc5af63f admin admin] RBAC: Authorizing `identity:delete_project(project_id=29510dc53bbb4201964ea85666d9e30b)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.557408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfb150-8eb2-40db-b72f-a1f8fc5af63f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.565123 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:48.591239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.591879 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.592208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.592484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] PATH_INFO: `/v3/projects/1fb5607b33a646838b2f1d2d5ad569db` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.593808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fb5607b33a646838b2f1d2d5ad569db)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.595848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.610214 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-debfb150-8eb2-40db-b72f-a1f8fc5af63f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29510dc53bbb4201964ea85666d9e30b', 'request_id': 'req-debfb150-8eb2-40db-b72f-a1f8fc5af63f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.618046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-debfb150-8eb2-40db-b72f-a1f8fc5af63f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29510dc53bbb4201964ea85666d9e30b', 'request_id': 'req-debfb150-8eb2-40db-b72f-a1f8fc5af63f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.626178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-debfb150-8eb2-40db-b72f-a1f8fc5af63f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29510dc53bbb4201964ea85666d9e30b', 'request_id': 'req-debfb150-8eb2-40db-b72f-a1f8fc5af63f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.637583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-debfb150-8eb2-40db-b72f-a1f8fc5af63f admin admin] The token cache is being invalidate because project 29510dc53bbb4201964ea85666d9e30b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:48.637922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-debfb150-8eb2-40db-b72f-a1f8fc5af63f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.638550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-debfb150-8eb2-40db-b72f-a1f8fc5af63f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.640076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1073/2130] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:48 2022] DELETE /identity/v3/projects/29510dc53bbb4201964ea85666d9e30b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:48.645961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16d6e5-0a19-4fd7-9443-91cf5dca857b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.646629 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16d6e5-0a19-4fd7-9443-91cf5dca857b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.646930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16d6e5-0a19-4fd7-9443-91cf5dca857b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.647231 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fb5607b33a646838b2f1d2d5ad569db', 'request_id': 'req-75149139-12e7-42e8-bcae-9d00fc4c15ca'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.658451 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fb5607b33a646838b2f1d2d5ad569db', 'request_id': 'req-75149139-12e7-42e8-bcae-9d00fc4c15ca'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.667415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fb5607b33a646838b2f1d2d5ad569db', 'request_id': 'req-75149139-12e7-42e8-bcae-9d00fc4c15ca'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.672096 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-fa16d6e5-0a19-4fd7-9443-91cf5dca857b None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:48.673476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fa16d6e5-0a19-4fd7-9443-91cf5dca857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:48.687015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] The token cache is being invalidate because project 1fb5607b33a646838b2f1d2d5ad569db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:48.687864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.689338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.691695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1058/2131] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:48 2022] DELETE /identity/v3/projects/1fb5607b33a646838b2f1d2d5ad569db => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:48.695883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.697481 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:48.756778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1074/2132] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:48.759112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dbd44-ad3d-41d3-909a-dde0bf80e383 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.759607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dbd44-ad3d-41d3-909a-dde0bf80e383 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.759949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dbd44-ad3d-41d3-909a-dde0bf80e383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.768026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75149139-12e7-42e8-bcae-9d00fc4c15ca admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.774302 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.774930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.775191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.775460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] PATH_INFO: `/v3/users/a676c03833e9418e87cf0352f4c1edd7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.776636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a676c03833e9418e87cf0352f4c1edd7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.778685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.789040 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a70dbd44-ad3d-41d3-909a-dde0bf80e383 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:48.789989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a70dbd44-ad3d-41d3-909a-dde0bf80e383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:48.799634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1075/2133] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:48.804171 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70dbd44-ad3d-41d3-909a-dde0bf80e383 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.808771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:48.818295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70dbd44-ad3d-41d3-909a-dde0bf80e383 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.826138 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9e02b-a9ab-4816-b546-8676d5f7b3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.828889 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9e02b-a9ab-4816-b546-8676d5f7b3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.829324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9e02b-a9ab-4816-b546-8676d5f7b3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.829465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9e02b-a9ab-4816-b546-8676d5f7b3d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.830157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9e02b-a9ab-4816-b546-8676d5f7b3d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:48.830489 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9e02b-a9ab-4816-b546-8676d5f7b3d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.832103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9e02b-a9ab-4816-b546-8676d5f7b3d5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.844401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1076/2134] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:48.845298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a676c03833e9418e87cf0352f4c1edd7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.846125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a676c03833e9418e87cf0352f4c1edd7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.846844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a676c03833e9418e87cf0352f4c1edd7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.848481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9e02b-a9ab-4816-b546-8676d5f7b3d5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.850253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d0c9e02b-a9ab-4816-b546-8676d5f7b3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:48.859840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a676c03833e9418e87cf0352f4c1edd7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.862845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9e02b-a9ab-4816-b546-8676d5f7b3d5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.863611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:48.868211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dea84-7043-4d35-bd22-838f8ad5f226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.868831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dea84-7043-4d35-bd22-838f8ad5f226 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.869226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dea84-7043-4d35-bd22-838f8ad5f226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.869500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dea84-7043-4d35-bd22-838f8ad5f226 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.870143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dea84-7043-4d35-bd22-838f8ad5f226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.872327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dea84-7043-4d35-bd22-838f8ad5f226 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.873428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a676c03833e9418e87cf0352f4c1edd7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.893280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1059/2135] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:48 2022] DELETE /identity/v3/users/a676c03833e9418e87cf0352f4c1edd7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:48.911195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1077/2136] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:48 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:48.912585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.003706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50012a89-503f-4814-9980-25ca6996e8c6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.076701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc25d6-99cd-4325-9a76-d111d92f5ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.078167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc25d6-99cd-4325-9a76-d111d92f5ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.078561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc25d6-99cd-4325-9a76-d111d92f5ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.078864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc25d6-99cd-4325-9a76-d111d92f5ce5 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.079484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc25d6-99cd-4325-9a76-d111d92f5ce5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.081864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc25d6-99cd-4325-9a76-d111d92f5ce5 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.116422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1060/2137] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:48 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:49.117913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dea84-7043-4d35-bd22-838f8ad5f226 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.128787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dea84-7043-4d35-bd22-838f8ad5f226 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.133417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134830f-3abd-4497-a594-8dbe90364c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.134022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134830f-3abd-4497-a594-8dbe90364c17 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.134300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134830f-3abd-4497-a594-8dbe90364c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.134568 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134830f-3abd-4497-a594-8dbe90364c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.135517 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134830f-3abd-4497-a594-8dbe90364c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.136849 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134830f-3abd-4497-a594-8dbe90364c17 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.148167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1078/2138] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:49.152283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc25d6-99cd-4325-9a76-d111d92f5ce5 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.165130 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc25d6-99cd-4325-9a76-d111d92f5ce5 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.171711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3b2ba-3818-4c37-a555-c9b90552e637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.172692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3b2ba-3818-4c37-a555-c9b90552e637 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.173088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3b2ba-3818-4c37-a555-c9b90552e637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.173456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3b2ba-3818-4c37-a555-c9b90552e637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.174256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3b2ba-3818-4c37-a555-c9b90552e637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.176026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3b2ba-3818-4c37-a555-c9b90552e637 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.188998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1061/2139] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:49.193171 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134830f-3abd-4497-a594-8dbe90364c17 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.203801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134830f-3abd-4497-a594-8dbe90364c17 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.208669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3e837-957b-4c3a-8d6c-5612e5858a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.209350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3e837-957b-4c3a-8d6c-5612e5858a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.209629 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3e837-957b-4c3a-8d6c-5612e5858a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.209905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3e837-957b-4c3a-8d6c-5612e5858a03 admin admin] PATH_INFO: `/v3/projects/dced2a3bcd8c4eeda9cf9ba25a3c9d23/users/f1387dc8257544b98437c2c242384872/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.214158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3b2ba-3818-4c37-a555-c9b90552e637 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.216188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b0e3b2ba-3818-4c37-a555-c9b90552e637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.229444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3e837-957b-4c3a-8d6c-5612e5858a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dced2a3bcd8c4eeda9cf9ba25a3c9d23, user_id=f1387dc8257544b98437c2c242384872, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.232474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3e837-957b-4c3a-8d6c-5612e5858a03 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.232757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3b2ba-3818-4c37-a555-c9b90552e637 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.239345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b0b84-5def-4b36-aca5-4857d37a9a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.240298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b0b84-5def-4b36-aca5-4857d37a9a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.240688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b0b84-5def-4b36-aca5-4857d37a9a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.241097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b0b84-5def-4b36-aca5-4857d37a9a6b admin admin] PATH_INFO: `/v3/projects/bc4b85ee8b7c4c02b830569b4c015836` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.242814 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b0b84-5def-4b36-aca5-4857d37a9a6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc4b85ee8b7c4c02b830569b4c015836)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.245908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b0b84-5def-4b36-aca5-4857d37a9a6b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.247751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1079/2140] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:49 2022] PUT /identity/v3/projects/dced2a3bcd8c4eeda9cf9ba25a3c9d23/users/f1387dc8257544b98437c2c242384872/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:49.262200 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342e192-c0e4-41d4-8c01-a331c4dba658 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.262540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342e192-c0e4-41d4-8c01-a331c4dba658 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.262816 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342e192-c0e4-41d4-8c01-a331c4dba658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.294839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c342e192-c0e4-41d4-8c01-a331c4dba658 None None] MFA Rules not processed for user `f1387dc8257544b98437c2c242384872`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:49.294839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c342e192-c0e4-41d4-8c01-a331c4dba658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.310987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c60b0b84-5def-4b36-aca5-4857d37a9a6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc4b85ee8b7c4c02b830569b4c015836', 'request_id': 'req-c60b0b84-5def-4b36-aca5-4857d37a9a6b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.324058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c60b0b84-5def-4b36-aca5-4857d37a9a6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc4b85ee8b7c4c02b830569b4c015836', 'request_id': 'req-c60b0b84-5def-4b36-aca5-4857d37a9a6b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.333966 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c60b0b84-5def-4b36-aca5-4857d37a9a6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc4b85ee8b7c4c02b830569b4c015836', 'request_id': 'req-c60b0b84-5def-4b36-aca5-4857d37a9a6b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.339397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:49.347759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c60b0b84-5def-4b36-aca5-4857d37a9a6b admin admin] The token cache is being invalidate because project bc4b85ee8b7c4c02b830569b4c015836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:49.348135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c60b0b84-5def-4b36-aca5-4857d37a9a6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.349102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c60b0b84-5def-4b36-aca5-4857d37a9a6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.354452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1062/2141] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:49 2022] DELETE /identity/v3/projects/bc4b85ee8b7c4c02b830569b4c015836 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:49.361112 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8538d-f86d-49cf-80f3-b1a6175ec9dc None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.361809 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8538d-f86d-49cf-80f3-b1a6175ec9dc None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.362243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8538d-f86d-49cf-80f3-b1a6175ec9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.397439 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-dfb8538d-f86d-49cf-80f3-b1a6175ec9dc None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:49.398867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dfb8538d-f86d-49cf-80f3-b1a6175ec9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.417159 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1080/2142] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:49.431709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342e192-c0e4-41d4-8c01-a331c4dba658 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.433165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c342e192-c0e4-41d4-8c01-a331c4dba658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.518048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1063/2143] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:49.523826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342e192-c0e4-41d4-8c01-a331c4dba658 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.528722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b535521-8936-4829-8e90-997cc4d7dd1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.529441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b535521-8936-4829-8e90-997cc4d7dd1f service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.529811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b535521-8936-4829-8e90-997cc4d7dd1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.530182 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b535521-8936-4829-8e90-997cc4d7dd1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.530310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779a146-3685-46ac-886d-a8d74e4dc391 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.530543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779a146-3685-46ac-886d-a8d74e4dc391 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.531816 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779a146-3685-46ac-886d-a8d74e4dc391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.540718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b535521-8936-4829-8e90-997cc4d7dd1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.542763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b535521-8936-4829-8e90-997cc4d7dd1f service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.554553 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1081/2144] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:49.579262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a779a146-3685-46ac-886d-a8d74e4dc391 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:49.579778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a779a146-3685-46ac-886d-a8d74e4dc391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.581199 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b535521-8936-4829-8e90-997cc4d7dd1f service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.582713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9b535521-8936-4829-8e90-997cc4d7dd1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.600294 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1064/2145] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:49.601742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779a146-3685-46ac-886d-a8d74e4dc391 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.610859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b535521-8936-4829-8e90-997cc4d7dd1f service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.612114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779a146-3685-46ac-886d-a8d74e4dc391 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.617157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaeee4-4ce2-4661-b89b-f277329befaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.617627 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaeee4-4ce2-4661-b89b-f277329befaf admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.617963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaeee4-4ce2-4661-b89b-f277329befaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.618231 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaeee4-4ce2-4661-b89b-f277329befaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.618908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaeee4-4ce2-4661-b89b-f277329befaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:49.619221 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaeee4-4ce2-4661-b89b-f277329befaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.621038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaeee4-4ce2-4661-b89b-f277329befaf admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.626400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.627281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.627554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.627962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] PATH_INFO: `/v3/users/f1387dc8257544b98437c2c242384872` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.630873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1387dc8257544b98437c2c242384872)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.632822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.637149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1065/2146] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:49.639167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaeee4-4ce2-4661-b89b-f277329befaf admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.652331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaeee4-4ce2-4661-b89b-f277329befaf admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.660424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2d04c-e497-4766-8910-009dad904213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.661075 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2d04c-e497-4766-8910-009dad904213 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.661395 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2d04c-e497-4766-8910-009dad904213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.661611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2d04c-e497-4766-8910-009dad904213 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.662244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2d04c-e497-4766-8910-009dad904213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.664164 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2d04c-e497-4766-8910-009dad904213 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.666516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:49.699023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1066/2147] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:49 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:49.701177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1387dc8257544b98437c2c242384872'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.703196 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1387dc8257544b98437c2c242384872'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.703968 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1387dc8257544b98437c2c242384872'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.708328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2d04c-e497-4766-8910-009dad904213 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.711240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bfc2d04c-e497-4766-8910-009dad904213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.726621 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1387dc8257544b98437c2c242384872'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.740437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1387dc8257544b98437c2c242384872'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.768392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1082/2148] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:49 2022] DELETE /identity/v3/users/f1387dc8257544b98437c2c242384872 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:49.773643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.775206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.791332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2d04c-e497-4766-8910-009dad904213 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.848331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249775fe-3ac0-48c4-8910-4874b51d53dc admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.853286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.853981 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.854259 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.854533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] PATH_INFO: `/v3/users/04e8f11d451e435595dc062e6d0e8300` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.855650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] RBAC: Authorizing `identity:delete_user(user_id=04e8f11d451e435595dc062e6d0e8300)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.858030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.861208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ea9e5-952a-4925-b06f-779c9dd240d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.861821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ea9e5-952a-4925-b06f-779c9dd240d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.862097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ea9e5-952a-4925-b06f-779c9dd240d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.862367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ea9e5-952a-4925-b06f-779c9dd240d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.862992 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ea9e5-952a-4925-b06f-779c9dd240d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.864938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ea9e5-952a-4925-b06f-779c9dd240d4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.873774 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:49.888040 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1067/2149] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:49 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:49.890049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdefce-bef4-49fd-881c-ecf884d0faf8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.890373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdefce-bef4-49fd-881c-ecf884d0faf8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.890661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdefce-bef4-49fd-881c-ecf884d0faf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.906580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04e8f11d451e435595dc062e6d0e8300'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.907303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04e8f11d451e435595dc062e6d0e8300'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.907993 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04e8f11d451e435595dc062e6d0e8300'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.911188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2dbdefce-bef4-49fd-881c-ecf884d0faf8 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:49.912411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2dbdefce-bef4-49fd-881c-ecf884d0faf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.920314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1068/2150] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:49.921117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdefce-bef4-49fd-881c-ecf884d0faf8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.921341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04e8f11d451e435595dc062e6d0e8300'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.925252 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2dbdefce-bef4-49fd-881c-ecf884d0faf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.931185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04e8f11d451e435595dc062e6d0e8300'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.939987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdefce-bef4-49fd-881c-ecf884d0faf8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.944094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1083/2151] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:49 2022] DELETE /identity/v3/users/04e8f11d451e435595dc062e6d0e8300 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:49.944931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.946261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.947507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed7107-91b4-4053-a8c6-123336f7e755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.948107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed7107-91b4-4053-a8c6-123336f7e755 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.948411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed7107-91b4-4053-a8c6-123336f7e755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.948680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed7107-91b4-4053-a8c6-123336f7e755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.949390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed7107-91b4-4053-a8c6-123336f7e755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.950959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed7107-91b4-4053-a8c6-123336f7e755 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.965709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1069/2152] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:49.965709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0a94d-2364-44f5-ac75-37a2bb34895b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.965709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0a94d-2364-44f5-ac75-37a2bb34895b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.965709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0a94d-2364-44f5-ac75-37a2bb34895b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.966135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:49.985687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8ef0a94d-2364-44f5-ac75-37a2bb34895b None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:49.987263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8ef0a94d-2364-44f5-ac75-37a2bb34895b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.012926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f24c93-1201-48f5-88b9-a8269e12aedc admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.017153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00c4db-98e8-4587-b82c-cd389f6f3cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.017763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00c4db-98e8-4587-b82c-cd389f6f3cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.018064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00c4db-98e8-4587-b82c-cd389f6f3cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.018354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00c4db-98e8-4587-b82c-cd389f6f3cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.052201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1070/2153] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:50.053289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0a94d-2364-44f5-ac75-37a2bb34895b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.064231 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0a94d-2364-44f5-ac75-37a2bb34895b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.069073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc49a21-451d-4bbb-8007-0e362a6e5b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.069768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc49a21-451d-4bbb-8007-0e362a6e5b5e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.070042 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc49a21-451d-4bbb-8007-0e362a6e5b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.070306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc49a21-451d-4bbb-8007-0e362a6e5b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.070973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc49a21-451d-4bbb-8007-0e362a6e5b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.072632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc49a21-451d-4bbb-8007-0e362a6e5b5e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.083207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1071/2154] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:50.085847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc49a21-451d-4bbb-8007-0e362a6e5b5e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.085962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00c4db-98e8-4587-b82c-cd389f6f3cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.087708 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00c4db-98e8-4587-b82c-cd389f6f3cc4 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.095821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc49a21-451d-4bbb-8007-0e362a6e5b5e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.097692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1084/2155] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:50.098518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00c4db-98e8-4587-b82c-cd389f6f3cc4 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.100616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ce9e7-9328-4522-b919-162c7a6aab71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.101311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ce9e7-9328-4522-b919-162c7a6aab71 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.101592 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ce9e7-9328-4522-b919-162c7a6aab71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.101859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ce9e7-9328-4522-b919-162c7a6aab71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.107652 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00c4db-98e8-4587-b82c-cd389f6f3cc4 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.111603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ce9e7-9328-4522-b919-162c7a6aab71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.113095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228665c-a892-4359-aeea-a346eaa276e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.113216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ce9e7-9328-4522-b919-162c7a6aab71 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.113757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228665c-a892-4359-aeea-a346eaa276e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.114037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228665c-a892-4359-aeea-a346eaa276e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.114347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228665c-a892-4359-aeea-a346eaa276e9 admin admin] PATH_INFO: `/v3/projects/4f43797a2b5a41b6b495c4cc89a2c699/users/9600a8cc7988495eb7c9e248d267ee6e/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.125843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1072/2156] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:50.130371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228665c-a892-4359-aeea-a346eaa276e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f43797a2b5a41b6b495c4cc89a2c699, user_id=9600a8cc7988495eb7c9e248d267ee6e, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.133380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228665c-a892-4359-aeea-a346eaa276e9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.149151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1085/2157] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:50 2022] PUT /identity/v3/projects/4f43797a2b5a41b6b495c4cc89a2c699/users/9600a8cc7988495eb7c9e248d267ee6e/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:50.161536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a57601-ce38-4d1c-98ce-8f39da34b249 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.161803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a57601-ce38-4d1c-98ce-8f39da34b249 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.162076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a57601-ce38-4d1c-98ce-8f39da34b249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.195907 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-96a57601-ce38-4d1c-98ce-8f39da34b249 None None] MFA Rules not processed for user `9600a8cc7988495eb7c9e248d267ee6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:50.197311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-96a57601-ce38-4d1c-98ce-8f39da34b249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.302407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1073/2158] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:50.314920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228665c-a892-4359-aeea-a346eaa276e9 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.316907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b228665c-a892-4359-aeea-a346eaa276e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.377504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228665c-a892-4359-aeea-a346eaa276e9 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.381920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a89f0d-65df-4476-8dbc-1b34bd1ed976 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.382746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a89f0d-65df-4476-8dbc-1b34bd1ed976 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.383028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a89f0d-65df-4476-8dbc-1b34bd1ed976 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.383402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a89f0d-65df-4476-8dbc-1b34bd1ed976 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.393898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a89f0d-65df-4476-8dbc-1b34bd1ed976 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.395700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a89f0d-65df-4476-8dbc-1b34bd1ed976 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.405611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1086/2159] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:50.530693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a57601-ce38-4d1c-98ce-8f39da34b249 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.532925 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-96a57601-ce38-4d1c-98ce-8f39da34b249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.589261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:50.604522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a89f0d-65df-4476-8dbc-1b34bd1ed976 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.607883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-71a89f0d-65df-4476-8dbc-1b34bd1ed976 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.618184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a57601-ce38-4d1c-98ce-8f39da34b249 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.623820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.624275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.624716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.625052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] PATH_INFO: `/v3/projects/dced2a3bcd8c4eeda9cf9ba25a3c9d23` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.626900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] RBAC: Authorizing `identity:delete_project(project_id=dced2a3bcd8c4eeda9cf9ba25a3c9d23)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.629055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.635341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a89f0d-65df-4476-8dbc-1b34bd1ed976 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.649976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.650753 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.651064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.651347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] PATH_INFO: `/v3/projects/b9789ee76dde42c183b100d175ac02c4` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.656733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9789ee76dde42c183b100d175ac02c4)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.660090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.686336 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dced2a3bcd8c4eeda9cf9ba25a3c9d23', 'request_id': 'req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:50.696223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dced2a3bcd8c4eeda9cf9ba25a3c9d23', 'request_id': 'req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:50.708640 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dced2a3bcd8c4eeda9cf9ba25a3c9d23', 'request_id': 'req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:50.721287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:50.723840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] The token cache is being invalidate because project dced2a3bcd8c4eeda9cf9ba25a3c9d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:50.725800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:50.727827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:50.729588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1074/2160] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:50 2022] DELETE /identity/v3/projects/dced2a3bcd8c4eeda9cf9ba25a3c9d23 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:50.734731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9789ee76dde42c183b100d175ac02c4', 'request_id': 'req-081154ae-c239-4f73-b183-71dc7ac3cf47'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:50.752580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9789ee76dde42c183b100d175ac02c4', 'request_id': 'req-081154ae-c239-4f73-b183-71dc7ac3cf47'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:50.780102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9789ee76dde42c183b100d175ac02c4', 'request_id': 'req-081154ae-c239-4f73-b183-71dc7ac3cf47'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:50.802918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] The token cache is being invalidate because project b9789ee76dde42c183b100d175ac02c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:50.803255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:50.803963 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:50.809324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1087/2161] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:50 2022] DELETE /identity/v3/projects/b9789ee76dde42c183b100d175ac02c4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:50.812672 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.815316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.916389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df9dcc-313e-4a89-9fe1-8f2cee73c0df admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.995938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb686a2-a671-4f27-b9a4-0d599217d8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.996583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb686a2-a671-4f27-b9a4-0d599217d8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.996961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb686a2-a671-4f27-b9a4-0d599217d8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.997173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb686a2-a671-4f27-b9a4-0d599217d8a9 admin admin] PATH_INFO: `/v3/users/902b1e2e038e466ea930a44dcf77ce67` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.998298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb686a2-a671-4f27-b9a4-0d599217d8a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=902b1e2e038e466ea930a44dcf77ce67)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.000466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb686a2-a671-4f27-b9a4-0d599217d8a9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.022017 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:51.050242 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4cb686a2-a671-4f27-b9a4-0d599217d8a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902b1e2e038e466ea930a44dcf77ce67'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:51.050921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4cb686a2-a671-4f27-b9a4-0d599217d8a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902b1e2e038e466ea930a44dcf77ce67'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:51.051594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4cb686a2-a671-4f27-b9a4-0d599217d8a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902b1e2e038e466ea930a44dcf77ce67'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:51.064267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4cb686a2-a671-4f27-b9a4-0d599217d8a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902b1e2e038e466ea930a44dcf77ce67'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:51.072611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4cb686a2-a671-4f27-b9a4-0d599217d8a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902b1e2e038e466ea930a44dcf77ce67'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:51.086071 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1075/2162] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:50 2022] DELETE /identity/v3/users/902b1e2e038e466ea930a44dcf77ce67 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:51.373041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:51.374720 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:51.393908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081154ae-c239-4f73-b183-71dc7ac3cf47 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:51.462454 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229626f-ff78-45cb-bd21-eeb7a453efca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:51.463488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229626f-ff78-45cb-bd21-eeb7a453efca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.463819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229626f-ff78-45cb-bd21-eeb7a453efca admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.464432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229626f-ff78-45cb-bd21-eeb7a453efca admin admin] PATH_INFO: `/v3/projects/d2b193fb367746028aaebeee1b8165fe` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.466010 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229626f-ff78-45cb-bd21-eeb7a453efca admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2b193fb367746028aaebeee1b8165fe)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.467961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229626f-ff78-45cb-bd21-eeb7a453efca admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.521605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9229626f-ff78-45cb-bd21-eeb7a453efca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2b193fb367746028aaebeee1b8165fe', 'request_id': 'req-9229626f-ff78-45cb-bd21-eeb7a453efca'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:51.529925 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9229626f-ff78-45cb-bd21-eeb7a453efca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2b193fb367746028aaebeee1b8165fe', 'request_id': 'req-9229626f-ff78-45cb-bd21-eeb7a453efca'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:51.537247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9229626f-ff78-45cb-bd21-eeb7a453efca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2b193fb367746028aaebeee1b8165fe', 'request_id': 'req-9229626f-ff78-45cb-bd21-eeb7a453efca'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:51.557058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9229626f-ff78-45cb-bd21-eeb7a453efca admin admin] The token cache is being invalidate because project d2b193fb367746028aaebeee1b8165fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:51.558327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9229626f-ff78-45cb-bd21-eeb7a453efca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:51.559148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-9229626f-ff78-45cb-bd21-eeb7a453efca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:51.565353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1088/2163] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:51 2022] DELETE /identity/v3/projects/d2b193fb367746028aaebeee1b8165fe => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:51.567313 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cd08b-a288-4d6f-963e-8a9ac4b1e741 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.567661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cd08b-a288-4d6f-963e-8a9ac4b1e741 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.568273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cd08b-a288-4d6f-963e-8a9ac4b1e741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.589412 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5c1cd08b-a288-4d6f-963e-8a9ac4b1e741 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:51.590705 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5c1cd08b-a288-4d6f-963e-8a9ac4b1e741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:51.656821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1076/2164] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:51.664169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67c7af-3975-4c27-a3dc-dc8d3cfb2f0c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.664457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67c7af-3975-4c27-a3dc-dc8d3cfb2f0c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.664759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67c7af-3975-4c27-a3dc-dc8d3cfb2f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.685202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6f67c7af-3975-4c27-a3dc-dc8d3cfb2f0c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:51.686490 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6f67c7af-3975-4c27-a3dc-dc8d3cfb2f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:51.695132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1089/2165] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:51.699582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cd08b-a288-4d6f-963e-8a9ac4b1e741 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:51.709808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cd08b-a288-4d6f-963e-8a9ac4b1e741 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:51.714653 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289180a4-4395-4770-bfd0-91885278bb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:51.715241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289180a4-4395-4770-bfd0-91885278bb95 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.715510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289180a4-4395-4770-bfd0-91885278bb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.715776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289180a4-4395-4770-bfd0-91885278bb95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.716454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289180a4-4395-4770-bfd0-91885278bb95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:51.716781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289180a4-4395-4770-bfd0-91885278bb95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.718596 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289180a4-4395-4770-bfd0-91885278bb95 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.730911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1077/2166] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:51.737325 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67c7af-3975-4c27-a3dc-dc8d3cfb2f0c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:51.747781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67c7af-3975-4c27-a3dc-dc8d3cfb2f0c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:51.754497 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479bdee8-c2aa-4156-b31c-ed755aeaff3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:51.755226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bdee8-c2aa-4156-b31c-ed755aeaff3c admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.755506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bdee8-c2aa-4156-b31c-ed755aeaff3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.755776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bdee8-c2aa-4156-b31c-ed755aeaff3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.756451 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479bdee8-c2aa-4156-b31c-ed755aeaff3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.760326 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479bdee8-c2aa-4156-b31c-ed755aeaff3c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.792502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1090/2167] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:51 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:51.797103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289180a4-4395-4770-bfd0-91885278bb95 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:51.814206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289180a4-4395-4770-bfd0-91885278bb95 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:51.878142 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3e7aa-0405-4b64-b0a6-c293d2adec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:51.878753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3e7aa-0405-4b64-b0a6-c293d2adec25 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.879030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3e7aa-0405-4b64-b0a6-c293d2adec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.879297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3e7aa-0405-4b64-b0a6-c293d2adec25 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.879924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3e7aa-0405-4b64-b0a6-c293d2adec25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.881895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3e7aa-0405-4b64-b0a6-c293d2adec25 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.906348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1078/2168] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:51 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:51.910598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479bdee8-c2aa-4156-b31c-ed755aeaff3c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:51.922170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479bdee8-c2aa-4156-b31c-ed755aeaff3c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:51.926417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619f448-ff65-40d2-927c-8474aa655e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:51.927149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619f448-ff65-40d2-927c-8474aa655e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.927398 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619f448-ff65-40d2-927c-8474aa655e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.927670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619f448-ff65-40d2-927c-8474aa655e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.928314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619f448-ff65-40d2-927c-8474aa655e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.930013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619f448-ff65-40d2-927c-8474aa655e4f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.945415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1091/2169] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:51.947001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3e7aa-0405-4b64-b0a6-c293d2adec25 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:51.963515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3e7aa-0405-4b64-b0a6-c293d2adec25 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:51.969213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b337770-77c8-4963-add2-64f9b2dcab97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:51.969830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b337770-77c8-4963-add2-64f9b2dcab97 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.970117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b337770-77c8-4963-add2-64f9b2dcab97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.970388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b337770-77c8-4963-add2-64f9b2dcab97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.971053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b337770-77c8-4963-add2-64f9b2dcab97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.972686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b337770-77c8-4963-add2-64f9b2dcab97 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.984329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1079/2170] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:51.988633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619f448-ff65-40d2-927c-8474aa655e4f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:52.000105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619f448-ff65-40d2-927c-8474aa655e4f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:52.005605 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d580e-23db-4d03-8821-742e7d5b9ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:52.006190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d580e-23db-4d03-8821-742e7d5b9ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:52.006469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d580e-23db-4d03-8821-742e7d5b9ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:52.006740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d580e-23db-4d03-8821-742e7d5b9ea2 admin admin] PATH_INFO: `/v3/projects/773c870545ec4a3ebce6d9c106411119/users/628fe7ca06ec4e34a499e69c14bab329/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:52.023350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d580e-23db-4d03-8821-742e7d5b9ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=773c870545ec4a3ebce6d9c106411119, user_id=628fe7ca06ec4e34a499e69c14bab329, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:52.026292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d580e-23db-4d03-8821-742e7d5b9ea2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:52.028506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:52.047426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1092/2171] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:51 2022] PUT /identity/v3/projects/773c870545ec4a3ebce6d9c106411119/users/628fe7ca06ec4e34a499e69c14bab329/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:52.049080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b162a-2d95-44ae-a5d0-5d8af2464de3 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:52.049641 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b162a-2d95-44ae-a5d0-5d8af2464de3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:52.050111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b162a-2d95-44ae-a5d0-5d8af2464de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:52.058019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:52.083108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b40b162a-2d95-44ae-a5d0-5d8af2464de3 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:52.084499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b40b162a-2d95-44ae-a5d0-5d8af2464de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:52.157270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1080/2172] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:52.174071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d580e-23db-4d03-8821-742e7d5b9ea2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:52.176217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-640d580e-23db-4d03-8821-742e7d5b9ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:52.238865 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d580e-23db-4d03-8821-742e7d5b9ea2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:52.242522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ae424-00a3-4a6f-b178-24b9209d2696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:52.243443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ae424-00a3-4a6f-b178-24b9209d2696 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:52.243443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ae424-00a3-4a6f-b178-24b9209d2696 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:52.243654 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ae424-00a3-4a6f-b178-24b9209d2696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:52.253187 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ae424-00a3-4a6f-b178-24b9209d2696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:52.254653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ae424-00a3-4a6f-b178-24b9209d2696 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:52.265001 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1093/2173] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:53.305437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4dc9c-d1ef-45ab-a839-352a821a5207 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:53.305950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4dc9c-d1ef-45ab-a839-352a821a5207 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:53.306238 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4dc9c-d1ef-45ab-a839-352a821a5207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:53.336028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ebf4dc9c-d1ef-45ab-a839-352a821a5207 None None] MFA Rules not processed for user `628fe7ca06ec4e34a499e69c14bab329`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:53.337261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ebf4dc9c-d1ef-45ab-a839-352a821a5207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:53.417926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1081/2174] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:53.423076 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ae424-00a3-4a6f-b178-24b9209d2696 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:53.436664 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ae424-00a3-4a6f-b178-24b9209d2696 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:53.443707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada561f2-4f46-4d0c-829c-3da73a7fcaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:53.445432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada561f2-4f46-4d0c-829c-3da73a7fcaef admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:53.445829 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada561f2-4f46-4d0c-829c-3da73a7fcaef admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:53.446147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada561f2-4f46-4d0c-829c-3da73a7fcaef admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:53.447454 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada561f2-4f46-4d0c-829c-3da73a7fcaef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:53.449551 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada561f2-4f46-4d0c-829c-3da73a7fcaef admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:53.480093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1094/2175] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:53 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:53.485028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4dc9c-d1ef-45ab-a839-352a821a5207 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:53.497319 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4dc9c-d1ef-45ab-a839-352a821a5207 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:53.556228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098f679-fb5b-4ba8-9d62-c29eb9f21a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:53.556890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098f679-fb5b-4ba8-9d62-c29eb9f21a74 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:53.557167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098f679-fb5b-4ba8-9d62-c29eb9f21a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:53.557436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098f679-fb5b-4ba8-9d62-c29eb9f21a74 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:53.558064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098f679-fb5b-4ba8-9d62-c29eb9f21a74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:53.560019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098f679-fb5b-4ba8-9d62-c29eb9f21a74 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:53.587673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1082/2176] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:53 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:53.593911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada561f2-4f46-4d0c-829c-3da73a7fcaef admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:53.609679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada561f2-4f46-4d0c-829c-3da73a7fcaef admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:53.614660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db3d2c-1c33-40f1-90a8-308312bdab7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:53.615278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db3d2c-1c33-40f1-90a8-308312bdab7e admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:53.615552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db3d2c-1c33-40f1-90a8-308312bdab7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:53.615818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db3d2c-1c33-40f1-90a8-308312bdab7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:53.616453 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db3d2c-1c33-40f1-90a8-308312bdab7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:53.618080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db3d2c-1c33-40f1-90a8-308312bdab7e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:53.628894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1095/2177] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:53.634636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098f679-fb5b-4ba8-9d62-c29eb9f21a74 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:53.647375 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098f679-fb5b-4ba8-9d62-c29eb9f21a74 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:53.653004 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de327315-f6ca-460a-ae21-ec44b3668600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:53.653654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de327315-f6ca-460a-ae21-ec44b3668600 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:53.653928 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de327315-f6ca-460a-ae21-ec44b3668600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:53.654197 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de327315-f6ca-460a-ae21-ec44b3668600 admin admin] PATH_INFO: `/v3/projects/a5789c24f985450bad95fe4978c5ca65/users/8137bb62fd384f4181dffef7cccda305/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:53.671696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de327315-f6ca-460a-ae21-ec44b3668600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5789c24f985450bad95fe4978c5ca65, user_id=8137bb62fd384f4181dffef7cccda305, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:53.674887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de327315-f6ca-460a-ae21-ec44b3668600 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:53.685862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:53.692587 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1083/2178] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:53 2022] PUT /identity/v3/projects/a5789c24f985450bad95fe4978c5ca65/users/8137bb62fd384f4181dffef7cccda305/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:53.696739 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db3d2c-1c33-40f1-90a8-308312bdab7e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:53.707034 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db3d2c-1c33-40f1-90a8-308312bdab7e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:53.768340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8811f-69f5-45a3-b556-76d9b44fbdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:53.768965 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8811f-69f5-45a3-b556-76d9b44fbdda admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:53.769243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8811f-69f5-45a3-b556-76d9b44fbdda admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:53.769511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8811f-69f5-45a3-b556-76d9b44fbdda admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:53.770149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8811f-69f5-45a3-b556-76d9b44fbdda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:53.771743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8811f-69f5-45a3-b556-76d9b44fbdda admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:53.786684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1096/2179] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:53.791772 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de327315-f6ca-460a-ae21-ec44b3668600 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:53.804508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de327315-f6ca-460a-ae21-ec44b3668600 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:53.809842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4203c5-ac6c-4fbc-af6a-f21ece444074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:53.810488 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4203c5-ac6c-4fbc-af6a-f21ece444074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:53.810768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4203c5-ac6c-4fbc-af6a-f21ece444074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:53.811037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4203c5-ac6c-4fbc-af6a-f21ece444074 admin admin] PATH_INFO: `/v3/projects/a5789c24f985450bad95fe4978c5ca65/users/8137bb62fd384f4181dffef7cccda305/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:53.814028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4203c5-ac6c-4fbc-af6a-f21ece444074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5789c24f985450bad95fe4978c5ca65, user_id=8137bb62fd384f4181dffef7cccda305, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:53.816695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4203c5-ac6c-4fbc-af6a-f21ece444074 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:53.828497 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1084/2180] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:53 2022] PUT /identity/v3/projects/a5789c24f985450bad95fe4978c5ca65/users/8137bb62fd384f4181dffef7cccda305/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:54.544961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07070ed1-8a29-4d0e-93ad-c2b7ee555ac3 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:54.545752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07070ed1-8a29-4d0e-93ad-c2b7ee555ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:54.546274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07070ed1-8a29-4d0e-93ad-c2b7ee555ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:54.578763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-07070ed1-8a29-4d0e-93ad-c2b7ee555ac3 None None] MFA Rules not processed for user `8137bb62fd384f4181dffef7cccda305`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:54.580181 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-07070ed1-8a29-4d0e-93ad-c2b7ee555ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:54.634361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:54.662470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1097/2181] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:54.673926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4203c5-ac6c-4fbc-af6a-f21ece444074 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:54.675759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4d4203c5-ac6c-4fbc-af6a-f21ece444074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:54.738878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4203c5-ac6c-4fbc-af6a-f21ece444074 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:54.742542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fa016-e818-4a96-bafa-15aca786ae91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:54.743148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fa016-e818-4a96-bafa-15aca786ae91 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:54.743425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fa016-e818-4a96-bafa-15aca786ae91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:54.743693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fa016-e818-4a96-bafa-15aca786ae91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:54.802185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fa016-e818-4a96-bafa-15aca786ae91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:54.803632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fa016-e818-4a96-bafa-15aca786ae91 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:54.813449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1085/2182] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:00.446878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07070ed1-8a29-4d0e-93ad-c2b7ee555ac3 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:00.449111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-07070ed1-8a29-4d0e-93ad-c2b7ee555ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:00.460935 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07070ed1-8a29-4d0e-93ad-c2b7ee555ac3 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:00.464631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961766ed-0ce4-4923-8fd0-aeeb22b41cd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:00.465254 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961766ed-0ce4-4923-8fd0-aeeb22b41cd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.465532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961766ed-0ce4-4923-8fd0-aeeb22b41cd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.465799 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961766ed-0ce4-4923-8fd0-aeeb22b41cd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.475116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961766ed-0ce4-4923-8fd0-aeeb22b41cd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:00.476818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961766ed-0ce4-4923-8fd0-aeeb22b41cd8 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:00.486779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1098/2183] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:04.808568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=54,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:09.887466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fa016-e818-4a96-bafa-15aca786ae91 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:09.891286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4f0fa016-e818-4a96-bafa-15aca786ae91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:09.979115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fa016-e818-4a96-bafa-15aca786ae91 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.985201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda64fc-7220-4090-9498-c2d402e9861f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.985849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda64fc-7220-4090-9498-c2d402e9861f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.986127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda64fc-7220-4090-9498-c2d402e9861f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.986380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda64fc-7220-4090-9498-c2d402e9861f admin admin] PATH_INFO: `/v3/users/8e1c70920a8a43999c308c8f595b4528` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.987543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda64fc-7220-4090-9498-c2d402e9861f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e1c70920a8a43999c308c8f595b4528)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.989576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda64fc-7220-4090-9498-c2d402e9861f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.043688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9bda64fc-7220-4090-9498-c2d402e9861f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e1c70920a8a43999c308c8f595b4528'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:10.044304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9bda64fc-7220-4090-9498-c2d402e9861f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e1c70920a8a43999c308c8f595b4528'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:10.044953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9bda64fc-7220-4090-9498-c2d402e9861f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e1c70920a8a43999c308c8f595b4528'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:10.058035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9bda64fc-7220-4090-9498-c2d402e9861f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e1c70920a8a43999c308c8f595b4528'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:10.068994 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9bda64fc-7220-4090-9498-c2d402e9861f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e1c70920a8a43999c308c8f595b4528'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:10.084981 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1086/2184] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:09 2022] DELETE /identity/v3/users/8e1c70920a8a43999c308c8f595b4528 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:10.272423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961766ed-0ce4-4923-8fd0-aeeb22b41cd8 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.289482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961766ed-0ce4-4923-8fd0-aeeb22b41cd8 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.351608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79386a-2c52-4c71-8228-bd1ec48e1e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.352280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79386a-2c52-4c71-8228-bd1ec48e1e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.352526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79386a-2c52-4c71-8228-bd1ec48e1e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.352845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79386a-2c52-4c71-8228-bd1ec48e1e52 admin admin] PATH_INFO: `/v3/projects/a646ac1b653b49fb80c05ad567f29710` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.354106 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79386a-2c52-4c71-8228-bd1ec48e1e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a646ac1b653b49fb80c05ad567f29710)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.356245 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79386a-2c52-4c71-8228-bd1ec48e1e52 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.407534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fa79386a-2c52-4c71-8228-bd1ec48e1e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a646ac1b653b49fb80c05ad567f29710', 'request_id': 'req-fa79386a-2c52-4c71-8228-bd1ec48e1e52'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:10.415670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fa79386a-2c52-4c71-8228-bd1ec48e1e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a646ac1b653b49fb80c05ad567f29710', 'request_id': 'req-fa79386a-2c52-4c71-8228-bd1ec48e1e52'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:10.422496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fa79386a-2c52-4c71-8228-bd1ec48e1e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a646ac1b653b49fb80c05ad567f29710', 'request_id': 'req-fa79386a-2c52-4c71-8228-bd1ec48e1e52'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:10.434150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fa79386a-2c52-4c71-8228-bd1ec48e1e52 admin admin] The token cache is being invalidate because project a646ac1b653b49fb80c05ad567f29710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:10.434441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fa79386a-2c52-4c71-8228-bd1ec48e1e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:10.435096 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fa79386a-2c52-4c71-8228-bd1ec48e1e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:10.436891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1099/2185] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:10 2022] DELETE /identity/v3/projects/a646ac1b653b49fb80c05ad567f29710 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:10.442424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a16eb-1299-4db4-9c72-b1714262091a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.442714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a16eb-1299-4db4-9c72-b1714262091a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.442991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a16eb-1299-4db4-9c72-b1714262091a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.463731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-096a16eb-1299-4db4-9c72-b1714262091a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:10.464971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-096a16eb-1299-4db4-9c72-b1714262091a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:10.526982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1087/2186] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:10.535202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51397a11-6577-4965-bbf8-a2e14eeff64f None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.535491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51397a11-6577-4965-bbf8-a2e14eeff64f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.535764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51397a11-6577-4965-bbf8-a2e14eeff64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.556494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-51397a11-6577-4965-bbf8-a2e14eeff64f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:10.557771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-51397a11-6577-4965-bbf8-a2e14eeff64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:10.567194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1100/2187] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:10.571852 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a16eb-1299-4db4-9c72-b1714262091a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.584618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a16eb-1299-4db4-9c72-b1714262091a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.590000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa2daa-1afd-4be5-b510-039c4eea1582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.590567 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa2daa-1afd-4be5-b510-039c4eea1582 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.590842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa2daa-1afd-4be5-b510-039c4eea1582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.591104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa2daa-1afd-4be5-b510-039c4eea1582 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.591754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa2daa-1afd-4be5-b510-039c4eea1582 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:10.592053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa2daa-1afd-4be5-b510-039c4eea1582 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.593720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa2daa-1afd-4be5-b510-039c4eea1582 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.606303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1088/2188] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:10.611928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51397a11-6577-4965-bbf8-a2e14eeff64f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.621509 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51397a11-6577-4965-bbf8-a2e14eeff64f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.628051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9c5de-fa3c-40fc-ad06-17086423e7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.628673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9c5de-fa3c-40fc-ad06-17086423e7ee admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.628964 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9c5de-fa3c-40fc-ad06-17086423e7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.629220 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9c5de-fa3c-40fc-ad06-17086423e7ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.629843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9c5de-fa3c-40fc-ad06-17086423e7ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.631795 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9c5de-fa3c-40fc-ad06-17086423e7ee admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.663215 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1101/2189] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:10 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:10.667942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa2daa-1afd-4be5-b510-039c4eea1582 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.679479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa2daa-1afd-4be5-b510-039c4eea1582 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.683231 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:10.741031 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446520d4-afed-4976-b1fd-b624accb6933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.741626 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446520d4-afed-4976-b1fd-b624accb6933 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.741904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446520d4-afed-4976-b1fd-b624accb6933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.742165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446520d4-afed-4976-b1fd-b624accb6933 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.742776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446520d4-afed-4976-b1fd-b624accb6933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.744694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446520d4-afed-4976-b1fd-b624accb6933 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.769111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1089/2190] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:10 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:10.773373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9c5de-fa3c-40fc-ad06-17086423e7ee admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.784160 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9c5de-fa3c-40fc-ad06-17086423e7ee admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.789127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796ce78-affd-42be-a3d2-cb5dc14a1938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.789737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796ce78-affd-42be-a3d2-cb5dc14a1938 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.790017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796ce78-affd-42be-a3d2-cb5dc14a1938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.790334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796ce78-affd-42be-a3d2-cb5dc14a1938 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.791112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796ce78-affd-42be-a3d2-cb5dc14a1938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.792989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796ce78-affd-42be-a3d2-cb5dc14a1938 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.803936 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1102/2191] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:10.807991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446520d4-afed-4976-b1fd-b624accb6933 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.818366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446520d4-afed-4976-b1fd-b624accb6933 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.823334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05665333-50f7-448b-b2f3-cf3663999d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.823925 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05665333-50f7-448b-b2f3-cf3663999d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.824198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05665333-50f7-448b-b2f3-cf3663999d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.824466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05665333-50f7-448b-b2f3-cf3663999d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.825142 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05665333-50f7-448b-b2f3-cf3663999d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.826779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05665333-50f7-448b-b2f3-cf3663999d4c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.837927 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1090/2192] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:10.842442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796ce78-affd-42be-a3d2-cb5dc14a1938 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.853706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796ce78-affd-42be-a3d2-cb5dc14a1938 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.859848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b69978-51fd-4291-9e0b-c866d213a8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.860499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b69978-51fd-4291-9e0b-c866d213a8f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.860806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b69978-51fd-4291-9e0b-c866d213a8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.861072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b69978-51fd-4291-9e0b-c866d213a8f0 admin admin] PATH_INFO: `/v3/projects/9095a2f3c7d946a2b017115364dc6753/users/fd39206c155f4e1cb9f7f90fb5a10556/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.877280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b69978-51fd-4291-9e0b-c866d213a8f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9095a2f3c7d946a2b017115364dc6753, user_id=fd39206c155f4e1cb9f7f90fb5a10556, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.880019 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b69978-51fd-4291-9e0b-c866d213a8f0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.891817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:10.898897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1103/2193] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:10 2022] PUT /identity/v3/projects/9095a2f3c7d946a2b017115364dc6753/users/fd39206c155f4e1cb9f7f90fb5a10556/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:10.904380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdbd0c-7397-40a3-b914-188583383a5f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.904924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdbd0c-7397-40a3-b914-188583383a5f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.905110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdbd0c-7397-40a3-b914-188583383a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.924829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4cbdbd0c-7397-40a3-b914-188583383a5f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:10.926163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4cbdbd0c-7397-40a3-b914-188583383a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:10.992246 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1091/2194] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:11.002435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b69978-51fd-4291-9e0b-c866d213a8f0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:11.005056 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d5b69978-51fd-4291-9e0b-c866d213a8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:11.068416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b69978-51fd-4291-9e0b-c866d213a8f0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:11.072758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b650470-d0c8-4539-8609-9980cfaa0e9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:11.073372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b650470-d0c8-4539-8609-9980cfaa0e9d service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:11.073655 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b650470-d0c8-4539-8609-9980cfaa0e9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:11.073919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b650470-d0c8-4539-8609-9980cfaa0e9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:11.084388 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b650470-d0c8-4539-8609-9980cfaa0e9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:11.085967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b650470-d0c8-4539-8609-9980cfaa0e9d service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:11.096151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1104/2195] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:15.693292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23511b6a-1dee-4ed7-be56-249904de4004 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:15.693835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23511b6a-1dee-4ed7-be56-249904de4004 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:15.694074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23511b6a-1dee-4ed7-be56-249904de4004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:15.730059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-23511b6a-1dee-4ed7-be56-249904de4004 None None] MFA Rules not processed for user `fd39206c155f4e1cb9f7f90fb5a10556`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:15.731616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-23511b6a-1dee-4ed7-be56-249904de4004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:15.861810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1092/2196] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:15.893584 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b650470-d0c8-4539-8609-9980cfaa0e9d service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:15.897518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2b650470-d0c8-4539-8609-9980cfaa0e9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:16.038177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b650470-d0c8-4539-8609-9980cfaa0e9d service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:16.051481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c196d0-918e-417b-a0c4-7c326698f337 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:16.052172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c196d0-918e-417b-a0c4-7c326698f337 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:16.052463 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c196d0-918e-417b-a0c4-7c326698f337 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:16.052812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c196d0-918e-417b-a0c4-7c326698f337 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:16.063357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c196d0-918e-417b-a0c4-7c326698f337 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:16.065046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c196d0-918e-417b-a0c4-7c326698f337 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:16.081616 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1105/2197] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:16.370802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23511b6a-1dee-4ed7-be56-249904de4004 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:16.372633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-23511b6a-1dee-4ed7-be56-249904de4004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:16.388897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23511b6a-1dee-4ed7-be56-249904de4004 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:16.394985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:16.395642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:16.395921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:16.396190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] PATH_INFO: `/v3/users/9600a8cc7988495eb7c9e248d267ee6e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:16.397475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9600a8cc7988495eb7c9e248d267ee6e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:16.399382 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:16.412233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:16.464506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9600a8cc7988495eb7c9e248d267ee6e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.465597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9600a8cc7988495eb7c9e248d267ee6e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.466554 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9600a8cc7988495eb7c9e248d267ee6e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.485062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9600a8cc7988495eb7c9e248d267ee6e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.495720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9600a8cc7988495eb7c9e248d267ee6e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:16.530480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1093/2198] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:16 2022] DELETE /identity/v3/users/9600a8cc7988495eb7c9e248d267ee6e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:16.537080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc7ce2-9e20-404b-ae4a-cb847cb213ba None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:16.537592 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc7ce2-9e20-404b-ae4a-cb847cb213ba None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:16.538010 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc7ce2-9e20-404b-ae4a-cb847cb213ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:16.565964 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a1fc7ce2-9e20-404b-ae4a-cb847cb213ba None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:16.567092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a1fc7ce2-9e20-404b-ae4a-cb847cb213ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:16.649751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1106/2199] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:16.660535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:16.662051 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:16.742209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a5942c-f2a1-4e99-9ae1-bcb18d40c439 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:16.746147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9aaf53-fab0-4b0d-8dba-ce9cf4438f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:16.746833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9aaf53-fab0-4b0d-8dba-ce9cf4438f6d service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:16.747175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9aaf53-fab0-4b0d-8dba-ce9cf4438f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:16.747506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9aaf53-fab0-4b0d-8dba-ce9cf4438f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:16.762765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9aaf53-fab0-4b0d-8dba-ce9cf4438f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:16.764742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9aaf53-fab0-4b0d-8dba-ce9cf4438f6d service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:16.782291 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1094/2200] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:17.116418 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc7ce2-9e20-404b-ae4a-cb847cb213ba None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:17.118685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a1fc7ce2-9e20-404b-ae4a-cb847cb213ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:17.140764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc7ce2-9e20-404b-ae4a-cb847cb213ba None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:17.157043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd206b34-dce5-4898-b84d-81f3c98e81b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:17.157696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd206b34-dce5-4898-b84d-81f3c98e81b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:17.158093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd206b34-dce5-4898-b84d-81f3c98e81b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:17.158413 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd206b34-dce5-4898-b84d-81f3c98e81b7 admin admin] PATH_INFO: `/v3/projects/4f43797a2b5a41b6b495c4cc89a2c699` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:17.161251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd206b34-dce5-4898-b84d-81f3c98e81b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f43797a2b5a41b6b495c4cc89a2c699)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:17.165731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd206b34-dce5-4898-b84d-81f3c98e81b7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:17.217653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:17.300552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bd206b34-dce5-4898-b84d-81f3c98e81b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f43797a2b5a41b6b495c4cc89a2c699', 'request_id': 'req-bd206b34-dce5-4898-b84d-81f3c98e81b7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.321149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bd206b34-dce5-4898-b84d-81f3c98e81b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f43797a2b5a41b6b495c4cc89a2c699', 'request_id': 'req-bd206b34-dce5-4898-b84d-81f3c98e81b7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.334432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bd206b34-dce5-4898-b84d-81f3c98e81b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f43797a2b5a41b6b495c4cc89a2c699', 'request_id': 'req-bd206b34-dce5-4898-b84d-81f3c98e81b7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.373880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bd206b34-dce5-4898-b84d-81f3c98e81b7 admin admin] The token cache is being invalidate because project 4f43797a2b5a41b6b495c4cc89a2c699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:17.374403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bd206b34-dce5-4898-b84d-81f3c98e81b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.376006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bd206b34-dce5-4898-b84d-81f3c98e81b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:17.378422 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1107/2201] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:17 2022] DELETE /identity/v3/projects/4f43797a2b5a41b6b495c4cc89a2c699 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:17.383025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eb184-7f71-42a7-a5ee-978ea1f5e210 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:17.383316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eb184-7f71-42a7-a5ee-978ea1f5e210 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:17.383588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eb184-7f71-42a7-a5ee-978ea1f5e210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:17.433058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-345eb184-7f71-42a7-a5ee-978ea1f5e210 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:17.437607 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-345eb184-7f71-42a7-a5ee-978ea1f5e210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:17.593187 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1095/2202] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:17.600751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d895e-a2c7-46a5-869e-4f749b37f6bf None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:17.601415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d895e-a2c7-46a5-869e-4f749b37f6bf None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:17.601856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d895e-a2c7-46a5-869e-4f749b37f6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:17.660337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6e8d895e-a2c7-46a5-869e-4f749b37f6bf None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:17.661609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6e8d895e-a2c7-46a5-869e-4f749b37f6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:17.701115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1108/2203] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:17.705526 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eb184-7f71-42a7-a5ee-978ea1f5e210 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:17.723753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eb184-7f71-42a7-a5ee-978ea1f5e210 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:17.743474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502665d-5a4a-4288-9556-d98521c24df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:17.744161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502665d-5a4a-4288-9556-d98521c24df8 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:17.744374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502665d-5a4a-4288-9556-d98521c24df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:17.749258 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502665d-5a4a-4288-9556-d98521c24df8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:17.750201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502665d-5a4a-4288-9556-d98521c24df8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:17.750655 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502665d-5a4a-4288-9556-d98521c24df8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:17.752671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502665d-5a4a-4288-9556-d98521c24df8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:17.777203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1096/2204] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:17.781928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d895e-a2c7-46a5-869e-4f749b37f6bf None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:17.801285 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d895e-a2c7-46a5-869e-4f749b37f6bf None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:17.808341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82433b-ef2f-4209-99eb-e995184599b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:17.809695 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82433b-ef2f-4209-99eb-e995184599b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:17.809977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82433b-ef2f-4209-99eb-e995184599b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:17.810252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82433b-ef2f-4209-99eb-e995184599b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:17.810986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82433b-ef2f-4209-99eb-e995184599b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:17.813475 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82433b-ef2f-4209-99eb-e995184599b3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:17.847879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1109/2205] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:17 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:34:17.852108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502665d-5a4a-4288-9556-d98521c24df8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:17.866156 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502665d-5a4a-4288-9556-d98521c24df8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:17.884058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:17.917803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a9fb1-3518-4bd6-9bf7-23696e87ff99 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:17.917908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a9fb1-3518-4bd6-9bf7-23696e87ff99 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:17.918176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a9fb1-3518-4bd6-9bf7-23696e87ff99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:17.953457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-245a9fb1-3518-4bd6-9bf7-23696e87ff99 None None] MFA Rules not processed for user `fd39206c155f4e1cb9f7f90fb5a10556`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:17.955170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-245a9fb1-3518-4bd6-9bf7-23696e87ff99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:17.991660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c223a2-c849-47f2-aeba-f6901f6c7623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:17.992284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c223a2-c849-47f2-aeba-f6901f6c7623 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:17.992568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c223a2-c849-47f2-aeba-f6901f6c7623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:17.992870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c223a2-c849-47f2-aeba-f6901f6c7623 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:17.993521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c223a2-c849-47f2-aeba-f6901f6c7623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:17.995747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c223a2-c849-47f2-aeba-f6901f6c7623 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:18.030833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1097/2206] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:17 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:18.032074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c223a2-c849-47f2-aeba-f6901f6c7623 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:18.050885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c223a2-c849-47f2-aeba-f6901f6c7623 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:18.059664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c7c60-3c3b-4943-85a4-8484ee07aeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:18.060425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c7c60-3c3b-4943-85a4-8484ee07aeec admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.060982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c7c60-3c3b-4943-85a4-8484ee07aeec admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.061383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c7c60-3c3b-4943-85a4-8484ee07aeec admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.062484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c7c60-3c3b-4943-85a4-8484ee07aeec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:18.065243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c7c60-3c3b-4943-85a4-8484ee07aeec admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:18.081625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1098/2207] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:18.090324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c7c60-3c3b-4943-85a4-8484ee07aeec admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:18.103343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c7c60-3c3b-4943-85a4-8484ee07aeec admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:18.106504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1110/2208] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:18.111652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0647e-cdbb-44c9-988f-a75915e1dfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:18.112428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0647e-cdbb-44c9-988f-a75915e1dfa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.112866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0647e-cdbb-44c9-988f-a75915e1dfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.113238 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0647e-cdbb-44c9-988f-a75915e1dfa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.114026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0647e-cdbb-44c9-988f-a75915e1dfa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:18.115711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0647e-cdbb-44c9-988f-a75915e1dfa4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:18.127200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1099/2209] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:18.131231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a9fb1-3518-4bd6-9bf7-23696e87ff99 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:18.143480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a9fb1-3518-4bd6-9bf7-23696e87ff99 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:18.149050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c2c28-3074-424e-98a1-3f6bb910302e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:18.149817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c2c28-3074-424e-98a1-3f6bb910302e admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.150173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c2c28-3074-424e-98a1-3f6bb910302e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.150566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c2c28-3074-424e-98a1-3f6bb910302e admin admin] PATH_INFO: `/v3/projects/afbe3e35bbfb4b5c8111d5d1a895155e/users/6b4b7190683a43d9a352e510fe66429d/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.171035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c2c28-3074-424e-98a1-3f6bb910302e admin admin] RBAC: Authorizing `identity:create_grant(project_id=afbe3e35bbfb4b5c8111d5d1a895155e, user_id=6b4b7190683a43d9a352e510fe66429d, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:18.174124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c2c28-3074-424e-98a1-3f6bb910302e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:18.197732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1111/2210] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:18 2022] PUT /identity/v3/projects/afbe3e35bbfb4b5c8111d5d1a895155e/users/6b4b7190683a43d9a352e510fe66429d/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:18.211801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c9c88-a5bd-474c-acb4-3bdb16ee2afe None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.212991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c9c88-a5bd-474c-acb4-3bdb16ee2afe None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.212991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c9c88-a5bd-474c-acb4-3bdb16ee2afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.253104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3f2c9c88-a5bd-474c-acb4-3bdb16ee2afe None None] MFA Rules not processed for user `6b4b7190683a43d9a352e510fe66429d`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:18.254405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3f2c9c88-a5bd-474c-acb4-3bdb16ee2afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:18.334802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1100/2211] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:18.343353 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c2c28-3074-424e-98a1-3f6bb910302e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:18.345379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fa8c2c28-3074-424e-98a1-3f6bb910302e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:18.410164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c2c28-3074-424e-98a1-3f6bb910302e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:18.415141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b7341-e11d-4d32-a481-4d2f24bf42c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:18.415927 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b7341-e11d-4d32-a481-4d2f24bf42c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.416210 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b7341-e11d-4d32-a481-4d2f24bf42c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.416487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b7341-e11d-4d32-a481-4d2f24bf42c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.428225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b7341-e11d-4d32-a481-4d2f24bf42c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:18.430020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b7341-e11d-4d32-a481-4d2f24bf42c9 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:18.442467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1112/2212] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:18.471934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c9c88-a5bd-474c-acb4-3bdb16ee2afe None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:18.543789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c9c88-a5bd-474c-acb4-3bdb16ee2afe None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:18.550085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:18.550883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.551690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.551690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] PATH_INFO: `/v3/users/6b4b7190683a43d9a352e510fe66429d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.552854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b4b7190683a43d9a352e510fe66429d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:18.555350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:18.577475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:18.622884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b4b7190683a43d9a352e510fe66429d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:18.623792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b4b7190683a43d9a352e510fe66429d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:18.624702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b4b7190683a43d9a352e510fe66429d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:18.639766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b4b7190683a43d9a352e510fe66429d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:18.648454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b4b7190683a43d9a352e510fe66429d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:18.673781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1101/2213] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:18 2022] DELETE /identity/v3/users/6b4b7190683a43d9a352e510fe66429d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:18.678958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc4047-b30b-4ac1-8c1e-bce03e91e811 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.680048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc4047-b30b-4ac1-8c1e-bce03e91e811 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.680560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc4047-b30b-4ac1-8c1e-bce03e91e811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.689890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:18.705167 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-40dc4047-b30b-4ac1-8c1e-bce03e91e811 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:18.706431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-40dc4047-b30b-4ac1-8c1e-bce03e91e811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:18.788686 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1113/2214] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:18.803571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:18.805264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:18.880745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebcc1f-229f-4a44-ab5d-4e0f5c4c8520 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:18.886140 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e9bb4-4cd8-4e2e-8d3e-731ec38f583c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:18.886474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e9bb4-4cd8-4e2e-8d3e-731ec38f583c service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.886752 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e9bb4-4cd8-4e2e-8d3e-731ec38f583c service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.887018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e9bb4-4cd8-4e2e-8d3e-731ec38f583c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.900177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e9bb4-4cd8-4e2e-8d3e-731ec38f583c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:18.901669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e9bb4-4cd8-4e2e-8d3e-731ec38f583c service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:18.912905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1102/2215] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:19.425648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc4047-b30b-4ac1-8c1e-bce03e91e811 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:19.428159 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-40dc4047-b30b-4ac1-8c1e-bce03e91e811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:19.447093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc4047-b30b-4ac1-8c1e-bce03e91e811 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:19.454752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:19.455771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:19.458801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:19.459312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] PATH_INFO: `/v3/projects/afbe3e35bbfb4b5c8111d5d1a895155e` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:19.461542 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afbe3e35bbfb4b5c8111d5d1a895155e)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:19.465042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:19.536561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afbe3e35bbfb4b5c8111d5d1a895155e', 'request_id': 'req-a3a129ce-7dfc-4638-be80-0393a1e695f4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:19.548112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afbe3e35bbfb4b5c8111d5d1a895155e', 'request_id': 'req-a3a129ce-7dfc-4638-be80-0393a1e695f4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:19.557896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afbe3e35bbfb4b5c8111d5d1a895155e', 'request_id': 'req-a3a129ce-7dfc-4638-be80-0393a1e695f4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:19.572198 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] The token cache is being invalidate because project afbe3e35bbfb4b5c8111d5d1a895155e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:19.572489 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:19.573364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:19.575356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1114/2216] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:19 2022] DELETE /identity/v3/projects/afbe3e35bbfb4b5c8111d5d1a895155e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:20.897047 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e9bb4-4cd8-4e2e-8d3e-731ec38f583c service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:20.898752 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ed1e9bb4-4cd8-4e2e-8d3e-731ec38f583c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:20.923317 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e9bb4-4cd8-4e2e-8d3e-731ec38f583c service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:21.031545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:21.032193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:21.032469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:21.033542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] PATH_INFO: `/v3/users/628fe7ca06ec4e34a499e69c14bab329` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:21.034043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=628fe7ca06ec4e34a499e69c14bab329)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:21.035992 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:21.090527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:21.095497 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '628fe7ca06ec4e34a499e69c14bab329'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.096358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '628fe7ca06ec4e34a499e69c14bab329'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.097521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '628fe7ca06ec4e34a499e69c14bab329'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.112545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '628fe7ca06ec4e34a499e69c14bab329'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.120291 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '628fe7ca06ec4e34a499e69c14bab329'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.133407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1103/2217] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:20 2022] DELETE /identity/v3/users/628fe7ca06ec4e34a499e69c14bab329 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:21.551333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:21.553111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:21.586063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a129ce-7dfc-4638-be80-0393a1e695f4 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:21.678265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:21.679374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:21.679805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:21.680233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] PATH_INFO: `/v3/projects/773c870545ec4a3ebce6d9c106411119` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:21.681548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=773c870545ec4a3ebce6d9c106411119)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:21.683531 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:21.721380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:21.735219 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '773c870545ec4a3ebce6d9c106411119', 'request_id': 'req-cc8c47ec-b576-4105-87db-6d274aed6b99'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.743972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '773c870545ec4a3ebce6d9c106411119', 'request_id': 'req-cc8c47ec-b576-4105-87db-6d274aed6b99'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.750517 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '773c870545ec4a3ebce6d9c106411119', 'request_id': 'req-cc8c47ec-b576-4105-87db-6d274aed6b99'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.762290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] The token cache is being invalidate because project 773c870545ec4a3ebce6d9c106411119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:21.762479 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.763151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.765445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1115/2218] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:21 2022] DELETE /identity/v3/projects/773c870545ec4a3ebce6d9c106411119 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:21.769582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:21.771049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:21.785843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f88b55-d2f1-418a-9941-60e2c9c3d8b9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:21.846757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0b408-00a9-4e2a-b5b4-4c1a81304fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:21.847495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0b408-00a9-4e2a-b5b4-4c1a81304fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:21.847842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0b408-00a9-4e2a-b5b4-4c1a81304fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:21.848179 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0b408-00a9-4e2a-b5b4-4c1a81304fc6 admin admin] PATH_INFO: `/v3/users/8137bb62fd384f4181dffef7cccda305` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:21.849585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0b408-00a9-4e2a-b5b4-4c1a81304fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8137bb62fd384f4181dffef7cccda305)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:21.851598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0b408-00a9-4e2a-b5b4-4c1a81304fc6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:21.921953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7eb0b408-00a9-4e2a-b5b4-4c1a81304fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8137bb62fd384f4181dffef7cccda305'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.922629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7eb0b408-00a9-4e2a-b5b4-4c1a81304fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8137bb62fd384f4181dffef7cccda305'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.923227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7eb0b408-00a9-4e2a-b5b4-4c1a81304fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8137bb62fd384f4181dffef7cccda305'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.941267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7eb0b408-00a9-4e2a-b5b4-4c1a81304fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8137bb62fd384f4181dffef7cccda305'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.955408 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7eb0b408-00a9-4e2a-b5b4-4c1a81304fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8137bb62fd384f4181dffef7cccda305'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.979034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1104/2219] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:21 2022] DELETE /identity/v3/users/8137bb62fd384f4181dffef7cccda305 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:22.594374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:22.596137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:22.610717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c47ec-b576-4105-87db-6d274aed6b99 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:22.687265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:22.691824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:22.692113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:22.692388 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad admin admin] PATH_INFO: `/v3/projects/a5789c24f985450bad95fe4978c5ca65` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:22.693745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5789c24f985450bad95fe4978c5ca65)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:22.695677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:22.744667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5789c24f985450bad95fe4978c5ca65', 'request_id': 'req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:22.753893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5789c24f985450bad95fe4978c5ca65', 'request_id': 'req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:22.761226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5789c24f985450bad95fe4978c5ca65', 'request_id': 'req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:22.772463 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad admin admin] The token cache is being invalidate because project a5789c24f985450bad95fe4978c5ca65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:22.772782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:22.773437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-d42d888b-b0b6-4acc-8e17-d0ef23c2f4ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:22.774939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1116/2220] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:22 2022] DELETE /identity/v3/projects/a5789c24f985450bad95fe4978c5ca65 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:22.780523 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee590a-b508-4389-91af-b4b27107f579 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:22.781075 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee590a-b508-4389-91af-b4b27107f579 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:22.781855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee590a-b508-4389-91af-b4b27107f579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:22.802595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-66ee590a-b508-4389-91af-b4b27107f579 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:22.803850 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-66ee590a-b508-4389-91af-b4b27107f579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:22.837346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:22.887530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1105/2221] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:22.901710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722a70f-dbb3-4bb4-8419-335c5b6e3a33 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:22.902526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722a70f-dbb3-4bb4-8419-335c5b6e3a33 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:22.903084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722a70f-dbb3-4bb4-8419-335c5b6e3a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:22.932989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6722a70f-dbb3-4bb4-8419-335c5b6e3a33 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:22.934401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6722a70f-dbb3-4bb4-8419-335c5b6e3a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:22.945157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1117/2222] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:22.949442 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee590a-b508-4389-91af-b4b27107f579 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:22.961125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee590a-b508-4389-91af-b4b27107f579 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:22.969713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7468db-8230-492a-8cbe-fa095771480d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:22.970442 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7468db-8230-492a-8cbe-fa095771480d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:22.970794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7468db-8230-492a-8cbe-fa095771480d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:22.971138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7468db-8230-492a-8cbe-fa095771480d admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:22.971809 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7468db-8230-492a-8cbe-fa095771480d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:22.972127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7468db-8230-492a-8cbe-fa095771480d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:22.973835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7468db-8230-492a-8cbe-fa095771480d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:22.990446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1106/2223] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:34:22.994589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722a70f-dbb3-4bb4-8419-335c5b6e3a33 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:23.004138 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722a70f-dbb3-4bb4-8419-335c5b6e3a33 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:23.008881 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f606c-3571-4fdd-93ec-31fce3e007ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:23.009557 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f606c-3571-4fdd-93ec-31fce3e007ba admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:23.009843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f606c-3571-4fdd-93ec-31fce3e007ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:23.010117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f606c-3571-4fdd-93ec-31fce3e007ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:23.010749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f606c-3571-4fdd-93ec-31fce3e007ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:23.012679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f606c-3571-4fdd-93ec-31fce3e007ba admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:23.043163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1118/2224] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:22 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:23.047777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7468db-8230-492a-8cbe-fa095771480d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:23.067812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7468db-8230-492a-8cbe-fa095771480d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:23.129887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42091e29-455b-498b-b50d-622b9155f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:23.130154 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42091e29-455b-498b-b50d-622b9155f772 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:23.130390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42091e29-455b-498b-b50d-622b9155f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:23.131492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42091e29-455b-498b-b50d-622b9155f772 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:23.131492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42091e29-455b-498b-b50d-622b9155f772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:23.134011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42091e29-455b-498b-b50d-622b9155f772 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:23.166296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1107/2225] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:23 2022] POST /identity/v3/users => generated 284 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:34:23.168386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f606c-3571-4fdd-93ec-31fce3e007ba admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:23.180313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f606c-3571-4fdd-93ec-31fce3e007ba admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:23.189817 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56484a70-ad02-44f9-add7-c3b5fd9a2122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:23.190764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56484a70-ad02-44f9-add7-c3b5fd9a2122 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:23.191145 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56484a70-ad02-44f9-add7-c3b5fd9a2122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:23.191426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56484a70-ad02-44f9-add7-c3b5fd9a2122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:23.192113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56484a70-ad02-44f9-add7-c3b5fd9a2122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:23.194266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56484a70-ad02-44f9-add7-c3b5fd9a2122 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:23.210623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1119/2226] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:23.211184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42091e29-455b-498b-b50d-622b9155f772 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:23.222964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42091e29-455b-498b-b50d-622b9155f772 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:23.227578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c9ee1-7e90-486f-a9c1-d86e815205d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:23.228241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c9ee1-7e90-486f-a9c1-d86e815205d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:23.228524 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c9ee1-7e90-486f-a9c1-d86e815205d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:23.228827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c9ee1-7e90-486f-a9c1-d86e815205d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:23.229478 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c9ee1-7e90-486f-a9c1-d86e815205d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:23.231160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c9ee1-7e90-486f-a9c1-d86e815205d3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:23.251751 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1108/2227] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:23.256074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56484a70-ad02-44f9-add7-c3b5fd9a2122 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:23.269887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56484a70-ad02-44f9-add7-c3b5fd9a2122 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:23.275165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf7384-45e4-435c-81b8-efb174f1a6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:23.275835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf7384-45e4-435c-81b8-efb174f1a6da admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:23.276113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf7384-45e4-435c-81b8-efb174f1a6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:23.276388 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf7384-45e4-435c-81b8-efb174f1a6da admin admin] PATH_INFO: `/v3/projects/b36934c826654069955886ad0a34e294/users/3c92e1e72b8f411fab3a57dd3a368b9d/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:23.280480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:23.298740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf7384-45e4-435c-81b8-efb174f1a6da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36934c826654069955886ad0a34e294, user_id=3c92e1e72b8f411fab3a57dd3a368b9d, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:23.302929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf7384-45e4-435c-81b8-efb174f1a6da admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:23.325163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1120/2228] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:23 2022] PUT /identity/v3/projects/b36934c826654069955886ad0a34e294/users/3c92e1e72b8f411fab3a57dd3a368b9d/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:23.337340 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd66862-9c6c-4445-aa93-def655b208be None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:23.337897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd66862-9c6c-4445-aa93-def655b208be None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:23.338412 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd66862-9c6c-4445-aa93-def655b208be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:23.371858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2cd66862-9c6c-4445-aa93-def655b208be None None] MFA Rules not processed for user `3c92e1e72b8f411fab3a57dd3a368b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:23.373932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2cd66862-9c6c-4445-aa93-def655b208be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:23.500178 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1109/2229] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:23.517904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf7384-45e4-435c-81b8-efb174f1a6da admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:23.520030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-83bf7384-45e4-435c-81b8-efb174f1a6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:23.615235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf7384-45e4-435c-81b8-efb174f1a6da admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:23.619032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95020413-47c1-42c7-8f26-155398e5c540 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94bfed415d24a9cb596b1fa92d5df97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:23.619643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95020413-47c1-42c7-8f26-155398e5c540 service glance] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:23.619930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95020413-47c1-42c7-8f26-155398e5c540 service glance] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:23.620225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95020413-47c1-42c7-8f26-155398e5c540 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:23.629576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95020413-47c1-42c7-8f26-155398e5c540 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:23.631112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95020413-47c1-42c7-8f26-155398e5c540 service glance] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:23.642394 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1121/2230] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:33.496853 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:33.637127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:37.023215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd66862-9c6c-4445-aa93-def655b208be None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:37.024758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2cd66862-9c6c-4445-aa93-def655b208be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:37.097122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd66862-9c6c-4445-aa93-def655b208be None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:37.101875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:37.102505 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:37.102777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:37.103041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] PATH_INFO: `/v3/users/d76439e4a1624e508566094cfde5c64e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:37.104165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76439e4a1624e508566094cfde5c64e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:37.106133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:37.162447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76439e4a1624e508566094cfde5c64e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:37.163210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76439e4a1624e508566094cfde5c64e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:37.163905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76439e4a1624e508566094cfde5c64e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:37.176504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76439e4a1624e508566094cfde5c64e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:37.183785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76439e4a1624e508566094cfde5c64e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:37.202632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1110/2231] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:37 2022] DELETE /identity/v3/users/d76439e4a1624e508566094cfde5c64e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:37.207693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1cafd-07f9-466c-bcd4-0c4c29fadb64 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:37.208220 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1cafd-07f9-466c-bcd4-0c4c29fadb64 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:37.208298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1cafd-07f9-466c-bcd4-0c4c29fadb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:37.228586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-64b1cafd-07f9-466c-bcd4-0c4c29fadb64 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:37.229854 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-64b1cafd-07f9-466c-bcd4-0c4c29fadb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:37.312404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1122/2232] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:37.322563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:37.324152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:37.387898 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a5c07-3d16-4634-9c1a-cad7c871da90 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:37.392000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051135a-5105-4cfd-a4c3-2ae424413b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:37.392603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051135a-5105-4cfd-a4c3-2ae424413b52 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:37.392912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051135a-5105-4cfd-a4c3-2ae424413b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:37.393183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051135a-5105-4cfd-a4c3-2ae424413b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:37.402763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051135a-5105-4cfd-a4c3-2ae424413b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:37.404370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051135a-5105-4cfd-a4c3-2ae424413b52 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:37.414657 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1111/2233] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:37.726595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1cafd-07f9-466c-bcd4-0c4c29fadb64 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:37.728125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-64b1cafd-07f9-466c-bcd4-0c4c29fadb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:37.742072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1cafd-07f9-466c-bcd4-0c4c29fadb64 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:37.748102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f0e50-59f0-4406-af3f-7e9d33cd8c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:37.749253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f0e50-59f0-4406-af3f-7e9d33cd8c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:37.749474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f0e50-59f0-4406-af3f-7e9d33cd8c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:37.749750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f0e50-59f0-4406-af3f-7e9d33cd8c23 admin admin] PATH_INFO: `/v3/projects/ad98afd2f9c444fe96b8577bfcdb065f` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:37.750954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f0e50-59f0-4406-af3f-7e9d33cd8c23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad98afd2f9c444fe96b8577bfcdb065f)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:37.753372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f0e50-59f0-4406-af3f-7e9d33cd8c23 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:37.810791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-031f0e50-59f0-4406-af3f-7e9d33cd8c23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad98afd2f9c444fe96b8577bfcdb065f', 'request_id': 'req-031f0e50-59f0-4406-af3f-7e9d33cd8c23'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:37.822293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-031f0e50-59f0-4406-af3f-7e9d33cd8c23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad98afd2f9c444fe96b8577bfcdb065f', 'request_id': 'req-031f0e50-59f0-4406-af3f-7e9d33cd8c23'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:37.829575 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-031f0e50-59f0-4406-af3f-7e9d33cd8c23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad98afd2f9c444fe96b8577bfcdb065f', 'request_id': 'req-031f0e50-59f0-4406-af3f-7e9d33cd8c23'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:37.841859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-031f0e50-59f0-4406-af3f-7e9d33cd8c23 admin admin] The token cache is being invalidate because project ad98afd2f9c444fe96b8577bfcdb065f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:37.842291 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-031f0e50-59f0-4406-af3f-7e9d33cd8c23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:37.843104 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-031f0e50-59f0-4406-af3f-7e9d33cd8c23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:37.845223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1123/2234] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:37 2022] DELETE /identity/v3/projects/ad98afd2f9c444fe96b8577bfcdb065f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:37.850842 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690e230-1244-4796-89ca-5ecfe231a732 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:37.851153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690e230-1244-4796-89ca-5ecfe231a732 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:37.851422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690e230-1244-4796-89ca-5ecfe231a732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:37.873829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c690e230-1244-4796-89ca-5ecfe231a732 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:37.875004 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c690e230-1244-4796-89ca-5ecfe231a732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:37.894973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:37.946003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1112/2235] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:37.954502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10349859-73c8-42e9-94f9-e5fe8d169f58 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:37.954794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10349859-73c8-42e9-94f9-e5fe8d169f58 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:37.955072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10349859-73c8-42e9-94f9-e5fe8d169f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:37.976870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-10349859-73c8-42e9-94f9-e5fe8d169f58 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:37.978091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-10349859-73c8-42e9-94f9-e5fe8d169f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:37.986885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1124/2236] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:37.991273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690e230-1244-4796-89ca-5ecfe231a732 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:38.002135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690e230-1244-4796-89ca-5ecfe231a732 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:38.007332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90ea07-5fad-43ce-aa7d-2fe9016e24e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:38.007932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90ea07-5fad-43ce-aa7d-2fe9016e24e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:38.008211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90ea07-5fad-43ce-aa7d-2fe9016e24e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:38.008480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90ea07-5fad-43ce-aa7d-2fe9016e24e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:38.009177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90ea07-5fad-43ce-aa7d-2fe9016e24e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:38.009479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90ea07-5fad-43ce-aa7d-2fe9016e24e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:38.011451 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90ea07-5fad-43ce-aa7d-2fe9016e24e3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:38.024305 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1113/2237] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:38.029099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10349859-73c8-42e9-94f9-e5fe8d169f58 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:38.038834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10349859-73c8-42e9-94f9-e5fe8d169f58 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:38.043741 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9653d16-d035-4b36-bb94-9b554c08b5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:38.044406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9653d16-d035-4b36-bb94-9b554c08b5db admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:38.044861 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9653d16-d035-4b36-bb94-9b554c08b5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:38.045018 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9653d16-d035-4b36-bb94-9b554c08b5db admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:38.045581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9653d16-d035-4b36-bb94-9b554c08b5db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:38.047526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9653d16-d035-4b36-bb94-9b554c08b5db admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:38.082445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1125/2238] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:38 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:38.088314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90ea07-5fad-43ce-aa7d-2fe9016e24e3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:38.099698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90ea07-5fad-43ce-aa7d-2fe9016e24e3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:38.164388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164b5be-4ab9-4266-8023-23c23e7274d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:38.164942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164b5be-4ab9-4266-8023-23c23e7274d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:38.165262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164b5be-4ab9-4266-8023-23c23e7274d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:38.165556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164b5be-4ab9-4266-8023-23c23e7274d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:38.166208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164b5be-4ab9-4266-8023-23c23e7274d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:38.168455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164b5be-4ab9-4266-8023-23c23e7274d0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:38.194220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1114/2239] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:38 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:38.198714 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9653d16-d035-4b36-bb94-9b554c08b5db admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:38.210470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9653d16-d035-4b36-bb94-9b554c08b5db admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:38.215246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326ea64-e6b0-4e10-a3e8-98efa2d0b598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:38.215838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326ea64-e6b0-4e10-a3e8-98efa2d0b598 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:38.216688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326ea64-e6b0-4e10-a3e8-98efa2d0b598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:38.217001 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326ea64-e6b0-4e10-a3e8-98efa2d0b598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:38.217650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326ea64-e6b0-4e10-a3e8-98efa2d0b598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:38.219267 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326ea64-e6b0-4e10-a3e8-98efa2d0b598 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:38.231155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1126/2240] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:38.235589 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164b5be-4ab9-4266-8023-23c23e7274d0 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:38.246191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164b5be-4ab9-4266-8023-23c23e7274d0 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:38.251925 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72efdf-a12e-478e-afb5-90720d5bc0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:38.252527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72efdf-a12e-478e-afb5-90720d5bc0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:38.252840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72efdf-a12e-478e-afb5-90720d5bc0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:38.253121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72efdf-a12e-478e-afb5-90720d5bc0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:38.253760 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72efdf-a12e-478e-afb5-90720d5bc0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:38.255366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72efdf-a12e-478e-afb5-90720d5bc0e6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:38.267746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1115/2241] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:38.272441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326ea64-e6b0-4e10-a3e8-98efa2d0b598 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:38.283687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326ea64-e6b0-4e10-a3e8-98efa2d0b598 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:38.289992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7964980-1495-48a6-9930-502e85ddf060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:38.290729 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7964980-1495-48a6-9930-502e85ddf060 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:38.291006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7964980-1495-48a6-9930-502e85ddf060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:38.291279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7964980-1495-48a6-9930-502e85ddf060 admin admin] PATH_INFO: `/v3/projects/5dc78d8e41d843a2b085f8300a285a0e/users/3a145fea1ee64ab18b93f3bf47c3fb65/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:38.308797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7964980-1495-48a6-9930-502e85ddf060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc78d8e41d843a2b085f8300a285a0e, user_id=3a145fea1ee64ab18b93f3bf47c3fb65, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:38.312175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7964980-1495-48a6-9930-502e85ddf060 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:38.332254 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1127/2242] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:38 2022] PUT /identity/v3/projects/5dc78d8e41d843a2b085f8300a285a0e/users/3a145fea1ee64ab18b93f3bf47c3fb65/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:38.333870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dd367-8aa6-4701-82be-5d4ff78abb24 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:38.334168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dd367-8aa6-4701-82be-5d4ff78abb24 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:38.334433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dd367-8aa6-4701-82be-5d4ff78abb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:38.355209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7e4dd367-8aa6-4701-82be-5d4ff78abb24 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:38.356838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7e4dd367-8aa6-4701-82be-5d4ff78abb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:38.425131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1116/2243] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:38.437042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7964980-1495-48a6-9930-502e85ddf060 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:38.439800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d7964980-1495-48a6-9930-502e85ddf060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:38.466145 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:38.505700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7964980-1495-48a6-9930-502e85ddf060 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:38.509975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131fb8e-a3bd-4046-af35-84e7d886385d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:38.510588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131fb8e-a3bd-4046-af35-84e7d886385d service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:38.510866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131fb8e-a3bd-4046-af35-84e7d886385d service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:38.511142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131fb8e-a3bd-4046-af35-84e7d886385d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:38.521344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131fb8e-a3bd-4046-af35-84e7d886385d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:38.522796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131fb8e-a3bd-4046-af35-84e7d886385d service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:38.533105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1128/2244] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:42.924024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186aaf45-0e19-4d1b-ba16-7e1cc4542b17 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:42.925020 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186aaf45-0e19-4d1b-ba16-7e1cc4542b17 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:42.926753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186aaf45-0e19-4d1b-ba16-7e1cc4542b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:42.958975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-186aaf45-0e19-4d1b-ba16-7e1cc4542b17 None None] MFA Rules not processed for user `3a145fea1ee64ab18b93f3bf47c3fb65`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:42.960311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-186aaf45-0e19-4d1b-ba16-7e1cc4542b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:43.002139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:43.087185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1117/2245] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:43.093424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131fb8e-a3bd-4046-af35-84e7d886385d service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:43.107355 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131fb8e-a3bd-4046-af35-84e7d886385d service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:43.115666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583c4dd-21c7-4b08-b198-6d2522dcf09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:43.116953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583c4dd-21c7-4b08-b198-6d2522dcf09f admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:43.117666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583c4dd-21c7-4b08-b198-6d2522dcf09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:43.118025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583c4dd-21c7-4b08-b198-6d2522dcf09f admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:43.118887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583c4dd-21c7-4b08-b198-6d2522dcf09f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:43.121474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583c4dd-21c7-4b08-b198-6d2522dcf09f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:43.169823 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1129/2246] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:43 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:43.170878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186aaf45-0e19-4d1b-ba16-7e1cc4542b17 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:43.192683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186aaf45-0e19-4d1b-ba16-7e1cc4542b17 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:43.334618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6f7ea-8c67-414e-bbc5-5b2453152108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:43.335265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6f7ea-8c67-414e-bbc5-5b2453152108 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:43.335550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6f7ea-8c67-414e-bbc5-5b2453152108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:43.338519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6f7ea-8c67-414e-bbc5-5b2453152108 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:43.339234 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6f7ea-8c67-414e-bbc5-5b2453152108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:43.341345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6f7ea-8c67-414e-bbc5-5b2453152108 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:43.412492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1118/2247] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:43 2022] POST /identity/v3/users => generated 298 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:43.414703 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583c4dd-21c7-4b08-b198-6d2522dcf09f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:43.431269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583c4dd-21c7-4b08-b198-6d2522dcf09f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:43.439165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665bbf3-a822-4b57-884e-1245d902c0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:43.440077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665bbf3-a822-4b57-884e-1245d902c0a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:43.440464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665bbf3-a822-4b57-884e-1245d902c0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:43.440971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665bbf3-a822-4b57-884e-1245d902c0a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:43.441722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665bbf3-a822-4b57-884e-1245d902c0a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:43.444316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665bbf3-a822-4b57-884e-1245d902c0a6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:43.464608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1130/2248] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:34:43.465649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6f7ea-8c67-414e-bbc5-5b2453152108 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:43.478867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6f7ea-8c67-414e-bbc5-5b2453152108 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:43.494019 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17c814-9052-4274-adda-ab59b7d68a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:43.496078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17c814-9052-4274-adda-ab59b7d68a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:43.496578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17c814-9052-4274-adda-ab59b7d68a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:43.497984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17c814-9052-4274-adda-ab59b7d68a1a admin admin] PATH_INFO: `/v3/projects/57051d7f1c4f45ae818325105ec0d5be/users/b33d94b97d51408286b7916e8b2a2fa7/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:43.518219 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17c814-9052-4274-adda-ab59b7d68a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=57051d7f1c4f45ae818325105ec0d5be, user_id=b33d94b97d51408286b7916e8b2a2fa7, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:43.522847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17c814-9052-4274-adda-ab59b7d68a1a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:43.551325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1119/2249] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:43 2022] PUT /identity/v3/projects/57051d7f1c4f45ae818325105ec0d5be/users/b33d94b97d51408286b7916e8b2a2fa7/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:43.555672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665bbf3-a822-4b57-884e-1245d902c0a6 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:43.576764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665bbf3-a822-4b57-884e-1245d902c0a6 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:43.699401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299dfcc-6351-49bf-8fee-00331cd59ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:43.700530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299dfcc-6351-49bf-8fee-00331cd59ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:43.700827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299dfcc-6351-49bf-8fee-00331cd59ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:43.701107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299dfcc-6351-49bf-8fee-00331cd59ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:43.701779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299dfcc-6351-49bf-8fee-00331cd59ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:43.710356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299dfcc-6351-49bf-8fee-00331cd59ba3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:43.730033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1131/2250] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:43.734767 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17c814-9052-4274-adda-ab59b7d68a1a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:43.752383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17c814-9052-4274-adda-ab59b7d68a1a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:43.760304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73341a5b-300c-4455-bae8-b52b6a02f3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:43.761022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73341a5b-300c-4455-bae8-b52b6a02f3b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:43.761301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73341a5b-300c-4455-bae8-b52b6a02f3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:43.761570 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73341a5b-300c-4455-bae8-b52b6a02f3b6 admin admin] PATH_INFO: `/v3/projects/57051d7f1c4f45ae818325105ec0d5be/users/b33d94b97d51408286b7916e8b2a2fa7/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:43.763951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73341a5b-300c-4455-bae8-b52b6a02f3b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57051d7f1c4f45ae818325105ec0d5be, user_id=b33d94b97d51408286b7916e8b2a2fa7, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:43.767471 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73341a5b-300c-4455-bae8-b52b6a02f3b6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:43.779568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1120/2251] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:43 2022] PUT /identity/v3/projects/57051d7f1c4f45ae818325105ec0d5be/users/b33d94b97d51408286b7916e8b2a2fa7/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:49.035036 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b1c77-d9e0-46d2-a6f6-d6663c80ae2d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:49.035534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b1c77-d9e0-46d2-a6f6-d6663c80ae2d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:49.035645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b1c77-d9e0-46d2-a6f6-d6663c80ae2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:49.069557 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5a0b1c77-d9e0-46d2-a6f6-d6663c80ae2d None None] MFA Rules not processed for user `b33d94b97d51408286b7916e8b2a2fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:49.070997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5a0b1c77-d9e0-46d2-a6f6-d6663c80ae2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:49.145263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1132/2252] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:49.159774 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73341a5b-300c-4455-bae8-b52b6a02f3b6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:49.161800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-73341a5b-300c-4455-bae8-b52b6a02f3b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:49.253517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73341a5b-300c-4455-bae8-b52b6a02f3b6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:49.257311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9067f-44f9-4716-a095-72cb987fdbb8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94bfed415d24a9cb596b1fa92d5df97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:49.257955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9067f-44f9-4716-a095-72cb987fdbb8 service glance] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:49.258302 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9067f-44f9-4716-a095-72cb987fdbb8 service glance] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:49.258633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9067f-44f9-4716-a095-72cb987fdbb8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:49.297838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:49.348397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9067f-44f9-4716-a095-72cb987fdbb8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:49.349903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9067f-44f9-4716-a095-72cb987fdbb8 service glance] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:49.368829 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1121/2253] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:49.554118 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4f40e-3f34-47c1-93ad-bd934a65a4e4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:49.554395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4f40e-3f34-47c1-93ad-bd934a65a4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:49.554656 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4f40e-3f34-47c1-93ad-bd934a65a4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:49.575679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-31d4f40e-3f34-47c1-93ad-bd934a65a4e4 None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:49.577012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-31d4f40e-3f34-47c1-93ad-bd934a65a4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:49.582992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:49.648834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1133/2254] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:34:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:49.700762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9067f-44f9-4716-a095-72cb987fdbb8 service glance] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:49.702309 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-71d9067f-44f9-4716-a095-72cb987fdbb8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:49.791766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9067f-44f9-4716-a095-72cb987fdbb8 service glance] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:49.805493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74a648-cb23-491e-8595-57c556092078 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:49.806164 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74a648-cb23-491e-8595-57c556092078 service swift] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:49.806477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74a648-cb23-491e-8595-57c556092078 service swift] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:49.807099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74a648-cb23-491e-8595-57c556092078 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:49.818384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74a648-cb23-491e-8595-57c556092078 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:49.820085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74a648-cb23-491e-8595-57c556092078 service swift] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:49.830210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1122/2255] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:34:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:34:53.280975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669a192-8b0d-4d2e-ae7f-8aa675c6b492 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:53.281492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669a192-8b0d-4d2e-ae7f-8aa675c6b492 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:53.281606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669a192-8b0d-4d2e-ae7f-8aa675c6b492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:53.316519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-9669a192-8b0d-4d2e-ae7f-8aa675c6b492 None None] MFA Rules not processed for user `e34cbcf69b4b4828a3bca077fc8d0694`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:53.317834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9669a192-8b0d-4d2e-ae7f-8aa675c6b492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:53.457643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1134/2256] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:59.827247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=48 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:00.908102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a23f9-ef09-4c32-8ce7-b7300e457c3c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:00.908580 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a23f9-ef09-4c32-8ce7-b7300e457c3c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:00.908687 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a23f9-ef09-4c32-8ce7-b7300e457c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:00.933899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4f6a23f9-ef09-4c32-8ce7-b7300e457c3c None None] MFA Rules not processed for user `3a145fea1ee64ab18b93f3bf47c3fb65`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:00.935496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4f6a23f9-ef09-4c32-8ce7-b7300e457c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:00.943120 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1123/2257] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:35:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:01.115871 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ef74d-dd4b-4430-b93c-dfd46cee431d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:01.116437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ef74d-dd4b-4430-b93c-dfd46cee431d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:01.116552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ef74d-dd4b-4430-b93c-dfd46cee431d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:01.138026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d05ef74d-dd4b-4430-b93c-dfd46cee431d None None] MFA Rules not processed for user `b4a617fc13534a89ad50a5fdc7bf769a`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:01.139609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d05ef74d-dd4b-4430-b93c-dfd46cee431d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:01.204885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1135/2258] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:35:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:02.685892 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a23f9-ef09-4c32-8ce7-b7300e457c3c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:02.689768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4f6a23f9-ef09-4c32-8ce7-b7300e457c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:02.778998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a23f9-ef09-4c32-8ce7-b7300e457c3c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:02.783872 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:02.784557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:02.784799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:02.785253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] PATH_INFO: `/v3/users/3c92e1e72b8f411fab3a57dd3a368b9d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:02.786478 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c92e1e72b8f411fab3a57dd3a368b9d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:02.788817 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:02.843997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c92e1e72b8f411fab3a57dd3a368b9d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:02.844808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c92e1e72b8f411fab3a57dd3a368b9d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:02.845425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c92e1e72b8f411fab3a57dd3a368b9d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:02.859215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c92e1e72b8f411fab3a57dd3a368b9d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:02.868763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c92e1e72b8f411fab3a57dd3a368b9d'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:02.882778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1124/2259] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:35:02 2022] DELETE /identity/v3/users/3c92e1e72b8f411fab3a57dd3a368b9d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:35:02.887835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10b32e-3dda-472e-91bc-673e08a1d500 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:02.888320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10b32e-3dda-472e-91bc-673e08a1d500 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:02.888416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10b32e-3dda-472e-91bc-673e08a1d500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:02.910171 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-ef10b32e-3dda-472e-91bc-673e08a1d500 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:02.911414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ef10b32e-3dda-472e-91bc-673e08a1d500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:02.974631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1136/2260] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:35:02.984969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:02.987339 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:03.051256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267779b3-2dca-4ace-b52b-46128b84a771 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:03.055107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052f223-d1d3-4836-aef0-f43142b166c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:03.055699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052f223-d1d3-4836-aef0-f43142b166c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:03.055974 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052f223-d1d3-4836-aef0-f43142b166c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:03.056241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052f223-d1d3-4836-aef0-f43142b166c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:03.066976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2052f223-d1d3-4836-aef0-f43142b166c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:03.068440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2052f223-d1d3-4836-aef0-f43142b166c3 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:03.078528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1125/2261] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:03.388131 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10b32e-3dda-472e-91bc-673e08a1d500 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:03.390587 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-ef10b32e-3dda-472e-91bc-673e08a1d500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:03.404788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10b32e-3dda-472e-91bc-673e08a1d500 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:03.410352 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:03.411006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:03.411688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:03.411688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624 admin admin] PATH_INFO: `/v3/projects/b36934c826654069955886ad0a34e294` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:03.412963 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b36934c826654069955886ad0a34e294)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:03.414913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:03.416638 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:03.470766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b36934c826654069955886ad0a34e294', 'request_id': 'req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:03.479922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b36934c826654069955886ad0a34e294', 'request_id': 'req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:03.486797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b36934c826654069955886ad0a34e294', 'request_id': 'req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:03.497317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624 admin admin] The token cache is being invalidate because project b36934c826654069955886ad0a34e294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:35:03.497608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:03.498252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f9ec2abe-6ebc-41b2-a9b5-37cb1b92b624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:03.499810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1137/2262] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:35:03 2022] DELETE /identity/v3/projects/b36934c826654069955886ad0a34e294 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:35:03.505382 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70665f0d-e7b0-4c01-acbf-923c4a9ed9b0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:03.505665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70665f0d-e7b0-4c01-acbf-923c4a9ed9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:03.505936 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70665f0d-e7b0-4c01-acbf-923c4a9ed9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:03.525571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-70665f0d-e7b0-4c01-acbf-923c4a9ed9b0 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:03.526740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-70665f0d-e7b0-4c01-acbf-923c4a9ed9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:03.536961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:03.598952 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1126/2263] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:35:03.601626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4dddf-6afb-4cce-a799-b9e5586f2d5a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:03.601985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4dddf-6afb-4cce-a799-b9e5586f2d5a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:03.602331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4dddf-6afb-4cce-a799-b9e5586f2d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:03.624530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-cbc4dddf-6afb-4cce-a799-b9e5586f2d5a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:03.625923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cbc4dddf-6afb-4cce-a799-b9e5586f2d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:03.634255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1138/2264] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:03.638770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70665f0d-e7b0-4c01-acbf-923c4a9ed9b0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:03.649495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70665f0d-e7b0-4c01-acbf-923c4a9ed9b0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:03.655033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0f799-d873-434c-9a1e-77e12389177d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:03.655638 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0f799-d873-434c-9a1e-77e12389177d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:03.655902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0f799-d873-434c-9a1e-77e12389177d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:03.656169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0f799-d873-434c-9a1e-77e12389177d admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:03.656857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0f799-d873-434c-9a1e-77e12389177d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:35:03.657158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0f799-d873-434c-9a1e-77e12389177d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:03.658769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0f799-d873-434c-9a1e-77e12389177d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:03.678185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1127/2265] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:35:03.685998 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4dddf-6afb-4cce-a799-b9e5586f2d5a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:03.697423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4dddf-6afb-4cce-a799-b9e5586f2d5a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:03.704482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2290d-3672-4671-9c34-b2e0a795b201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:03.704482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2290d-3672-4671-9c34-b2e0a795b201 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:03.704482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2290d-3672-4671-9c34-b2e0a795b201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:03.704869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2290d-3672-4671-9c34-b2e0a795b201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:03.705493 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2290d-3672-4671-9c34-b2e0a795b201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:03.707715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2290d-3672-4671-9c34-b2e0a795b201 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:03.738685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1139/2266] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:35:03 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:35:03.747474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0f799-d873-434c-9a1e-77e12389177d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:03.758190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0f799-d873-434c-9a1e-77e12389177d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:03.829543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08794c6-ed99-4d64-8167-e03e83c77a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:03.830653 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08794c6-ed99-4d64-8167-e03e83c77a5b admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:03.831163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08794c6-ed99-4d64-8167-e03e83c77a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:03.831754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08794c6-ed99-4d64-8167-e03e83c77a5b admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:03.832447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08794c6-ed99-4d64-8167-e03e83c77a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:03.834619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08794c6-ed99-4d64-8167-e03e83c77a5b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:03.860689 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1128/2267] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:35:03 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:35:03.863555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2290d-3672-4671-9c34-b2e0a795b201 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:03.874131 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2290d-3672-4671-9c34-b2e0a795b201 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:03.878771 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0deec1f-dc13-4adf-a99f-ad3662a24d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:03.879380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0deec1f-dc13-4adf-a99f-ad3662a24d54 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:03.879659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0deec1f-dc13-4adf-a99f-ad3662a24d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:03.879931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0deec1f-dc13-4adf-a99f-ad3662a24d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:03.880579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0deec1f-dc13-4adf-a99f-ad3662a24d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:03.882221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0deec1f-dc13-4adf-a99f-ad3662a24d54 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:03.892749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1140/2268] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:35:03.896831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08794c6-ed99-4d64-8167-e03e83c77a5b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:03.907535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08794c6-ed99-4d64-8167-e03e83c77a5b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:03.912811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608422c6-a440-4525-a6be-9836daf62ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:03.913485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608422c6-a440-4525-a6be-9836daf62ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:03.913692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608422c6-a440-4525-a6be-9836daf62ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:03.913961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608422c6-a440-4525-a6be-9836daf62ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:03.914638 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608422c6-a440-4525-a6be-9836daf62ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:03.916245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608422c6-a440-4525-a6be-9836daf62ef4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:03.927646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1129/2269] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:35:03.931923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0deec1f-dc13-4adf-a99f-ad3662a24d54 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:03.942753 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0deec1f-dc13-4adf-a99f-ad3662a24d54 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:03.947573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43e776-f4c9-408c-a32f-0d4e8834b744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:03.948249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43e776-f4c9-408c-a32f-0d4e8834b744 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:03.948573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43e776-f4c9-408c-a32f-0d4e8834b744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:03.948971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43e776-f4c9-408c-a32f-0d4e8834b744 admin admin] PATH_INFO: `/v3/projects/e3e2e4f6da374afba391fec0fa0bb3fb/users/35d65aef527d421c979e400774fc3fee/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:03.963644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43e776-f4c9-408c-a32f-0d4e8834b744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e2e4f6da374afba391fec0fa0bb3fb, user_id=35d65aef527d421c979e400774fc3fee, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:03.966480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43e776-f4c9-408c-a32f-0d4e8834b744 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:03.981837 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1141/2270] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:35:03 2022] PUT /identity/v3/projects/e3e2e4f6da374afba391fec0fa0bb3fb/users/35d65aef527d421c979e400774fc3fee/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:35:03.986950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108817af-6dca-4dd0-ae3e-723cb0a21ec0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:03.987237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108817af-6dca-4dd0-ae3e-723cb0a21ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:03.987513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108817af-6dca-4dd0-ae3e-723cb0a21ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:04.017105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-108817af-6dca-4dd0-ae3e-723cb0a21ec0 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:04.018349 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-108817af-6dca-4dd0-ae3e-723cb0a21ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:04.079705 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1130/2271] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:04.092400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43e776-f4c9-408c-a32f-0d4e8834b744 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:04.095014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0f43e776-f4c9-408c-a32f-0d4e8834b744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:04.163752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43e776-f4c9-408c-a32f-0d4e8834b744 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:04.167973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebffabb-417b-44a7-8ec7-a133b4e21fbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:04.168579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebffabb-417b-44a7-8ec7-a133b4e21fbc service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:04.168964 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebffabb-417b-44a7-8ec7-a133b4e21fbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:04.169238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebffabb-417b-44a7-8ec7-a133b4e21fbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:04.178983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebffabb-417b-44a7-8ec7-a133b4e21fbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:04.180408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebffabb-417b-44a7-8ec7-a133b4e21fbc service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:04.191779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1142/2272] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:05.186806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ee14d-6f8c-401c-8d90-1c0f868b23d5 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:05.187710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ee14d-6f8c-401c-8d90-1c0f868b23d5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:05.188756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ee14d-6f8c-401c-8d90-1c0f868b23d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:05.212489 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6c6ee14d-6f8c-401c-8d90-1c0f868b23d5 None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:05.214658 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6c6ee14d-6f8c-401c-8d90-1c0f868b23d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:05.240631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:05.302028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1131/2273] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:35:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:05.315966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebffabb-417b-44a7-8ec7-a133b4e21fbc service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:05.317737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-aebffabb-417b-44a7-8ec7-a133b4e21fbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:05.355560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:05.413826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebffabb-417b-44a7-8ec7-a133b4e21fbc service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:05.419445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d3fd4-a0c6-4489-8fee-272e7580907d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:05.420059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d3fd4-a0c6-4489-8fee-272e7580907d service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:05.420338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d3fd4-a0c6-4489-8fee-272e7580907d service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:05.420610 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d3fd4-a0c6-4489-8fee-272e7580907d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:05.431172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d3fd4-a0c6-4489-8fee-272e7580907d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:05.433031 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d3fd4-a0c6-4489-8fee-272e7580907d service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:05.442617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1143/2274] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:35:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:35:06.281705 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cb326-5e9a-48f8-9f7a-5ff3b6407a85 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:06.286110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cb326-5e9a-48f8-9f7a-5ff3b6407a85 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:06.286110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cb326-5e9a-48f8-9f7a-5ff3b6407a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:06.350343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-dc0cb326-5e9a-48f8-9f7a-5ff3b6407a85 None None] MFA Rules not processed for user `35d65aef527d421c979e400774fc3fee`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:06.351746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dc0cb326-5e9a-48f8-9f7a-5ff3b6407a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:06.521370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1132/2275] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:35:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:06.539373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d3fd4-a0c6-4489-8fee-272e7580907d service swift] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:06.542309 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2b8d3fd4-a0c6-4489-8fee-272e7580907d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:06.695436 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d3fd4-a0c6-4489-8fee-272e7580907d service swift] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:06.702889 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d3a99-d29e-4248-bbce-67822aa3e6ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:06.703611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d3a99-d29e-4248-bbce-67822aa3e6ab service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:06.703961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d3a99-d29e-4248-bbce-67822aa3e6ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:06.704302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d3a99-d29e-4248-bbce-67822aa3e6ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:06.717888 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d3a99-d29e-4248-bbce-67822aa3e6ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:06.719791 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d3a99-d29e-4248-bbce-67822aa3e6ab service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:06.733128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1144/2276] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:08.322643 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f4b25-2ca0-45a9-961f-38af04e44523 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:08.323283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f4b25-2ca0-45a9-961f-38af04e44523 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:08.323283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f4b25-2ca0-45a9-961f-38af04e44523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:08.342133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c91f4b25-2ca0-45a9-961f-38af04e44523 None None] MFA Rules not processed for user `35d65aef527d421c979e400774fc3fee`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:08.343325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c91f4b25-2ca0-45a9-961f-38af04e44523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:08.350187 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1133/2277] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:35:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:16.727335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=45 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:18.336919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=50 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:44.519471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d3a99-d29e-4248-bbce-67822aa3e6ab service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:44.521272 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-910d3a99-d29e-4248-bbce-67822aa3e6ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:44.533921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d3a99-d29e-4248-bbce-67822aa3e6ab service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:44.538081 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e9f07-7598-490f-a7ce-7b31f3e6db1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:44.538668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e9f07-7598-490f-a7ce-7b31f3e6db1c service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:44.538941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e9f07-7598-490f-a7ce-7b31f3e6db1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:44.539175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e9f07-7598-490f-a7ce-7b31f3e6db1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:44.541342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-376e9f07-7598-490f-a7ce-7b31f3e6db1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:44.607516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e9f07-7598-490f-a7ce-7b31f3e6db1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:44.608966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e9f07-7598-490f-a7ce-7b31f3e6db1c service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:44.621111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1145/2278] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:53.403971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed2d7a-797f-41f5-aa73-d9a585e11e2d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:53.404534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed2d7a-797f-41f5-aa73-d9a585e11e2d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:53.404631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed2d7a-797f-41f5-aa73-d9a585e11e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:53.425115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-34ed2d7a-797f-41f5-aa73-d9a585e11e2d None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:53.426325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-34ed2d7a-797f-41f5-aa73-d9a585e11e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:53.435213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1134/2279] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:35:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:35:53.458561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e9f07-7598-490f-a7ce-7b31f3e6db1c service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:53.470703 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e9f07-7598-490f-a7ce-7b31f3e6db1c service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:53.476825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5df8cc-6bfc-4684-9754-2c5108c8ec04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:53.477424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5df8cc-6bfc-4684-9754-2c5108c8ec04 service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:53.477898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5df8cc-6bfc-4684-9754-2c5108c8ec04 service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:53.478217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5df8cc-6bfc-4684-9754-2c5108c8ec04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:53.487523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5df8cc-6bfc-4684-9754-2c5108c8ec04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:53.489155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5df8cc-6bfc-4684-9754-2c5108c8ec04 service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:53.497690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1146/2280] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:35:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:36:03.419694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:36:03.494503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=36 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:36:09.709582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed2d7a-797f-41f5-aa73-d9a585e11e2d None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:36:09.711075 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-34ed2d7a-797f-41f5-aa73-d9a585e11e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:09.793549 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed2d7a-797f-41f5-aa73-d9a585e11e2d None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:36:09.799405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da4ab0-903e-4025-9fd9-11ea5b937238 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:36:09.800044 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da4ab0-903e-4025-9fd9-11ea5b937238 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:09.800298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da4ab0-903e-4025-9fd9-11ea5b937238 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:09.800565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da4ab0-903e-4025-9fd9-11ea5b937238 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:09.802661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-24da4ab0-903e-4025-9fd9-11ea5b937238 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:09.815553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da4ab0-903e-4025-9fd9-11ea5b937238 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:36:09.817124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da4ab0-903e-4025-9fd9-11ea5b937238 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:36:09.828324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1135/2281] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:36:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:36:11.489207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511f5c7-120f-4c9d-9f63-31d26f8f9980 None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:11.489859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511f5c7-120f-4c9d-9f63-31d26f8f9980 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:11.489859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511f5c7-120f-4c9d-9f63-31d26f8f9980 None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:11.491079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1147/2282] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:36:11.497681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11ac11-c4df-4fe3-9910-288bf4c5c92f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:11.498215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11ac11-c4df-4fe3-9910-288bf4c5c92f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:11.498317 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11ac11-c4df-4fe3-9910-288bf4c5c92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:11.518868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1e11ac11-c4df-4fe3-9910-288bf4c5c92f None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:36:11.520159 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1e11ac11-c4df-4fe3-9910-288bf4c5c92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:11.591459 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1136/2283] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:36:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:36:11.604303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511f5c7-120f-4c9d-9f63-31d26f8f9980 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:36:11.606239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e511f5c7-120f-4c9d-9f63-31d26f8f9980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:11.667548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511f5c7-120f-4c9d-9f63-31d26f8f9980 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:36:11.671374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74a082-cdcc-4b72-9c0d-b27e330c0c0f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94bfed415d24a9cb596b1fa92d5df97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:36:11.671980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74a082-cdcc-4b72-9c0d-b27e330c0c0f service glance] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:11.672257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74a082-cdcc-4b72-9c0d-b27e330c0c0f service glance] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:11.672528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74a082-cdcc-4b72-9c0d-b27e330c0c0f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:11.683550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74a082-cdcc-4b72-9c0d-b27e330c0c0f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:36:11.685021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74a082-cdcc-4b72-9c0d-b27e330c0c0f service glance] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:36:11.696060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1148/2284] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:36:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:36:11.782684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b909e0-64b3-4f18-be75-f9bc05614169 None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:11.782971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b909e0-64b3-4f18-be75-f9bc05614169 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:11.783254 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b909e0-64b3-4f18-be75-f9bc05614169 None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:11.784750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1137/2285] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:36:11.790499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9343d-5b6c-4ebd-aed5-15d1deda9deb None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:11.790960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9343d-5b6c-4ebd-aed5-15d1deda9deb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:11.791394 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9343d-5b6c-4ebd-aed5-15d1deda9deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:11.816818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-20a9343d-5b6c-4ebd-aed5-15d1deda9deb None None] MFA Rules not processed for user `5e18461c8cc4497eae7a83d9ad1448f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:36:11.818037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-20a9343d-5b6c-4ebd-aed5-15d1deda9deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:11.826812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1149/2286] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:36:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:36:11.840541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b909e0-64b3-4f18-be75-f9bc05614169 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:36:11.852503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b909e0-64b3-4f18-be75-f9bc05614169 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:36:11.856497 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78326f2b-3245-4564-9011-8147c8f729aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:36:11.857116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78326f2b-3245-4564-9011-8147c8f729aa service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:11.857391 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78326f2b-3245-4564-9011-8147c8f729aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:11.857656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78326f2b-3245-4564-9011-8147c8f729aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:11.866119 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78326f2b-3245-4564-9011-8147c8f729aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:36:11.868103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78326f2b-3245-4564-9011-8147c8f729aa service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:36:11.875791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1138/2287] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:36:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:36:11.964890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db27c1-0b13-4596-ae03-e05ebba56f0f None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:11.965181 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db27c1-0b13-4596-ae03-e05ebba56f0f None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:11.965459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db27c1-0b13-4596-ae03-e05ebba56f0f None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:11.966898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1150/2288] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:36:11.972903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e6e75-5dde-4ec9-9c9b-d6e4a072fca7 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:11.973165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e6e75-5dde-4ec9-9c9b-d6e4a072fca7 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:11.973537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e6e75-5dde-4ec9-9c9b-d6e4a072fca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:11.996621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1d8e6e75-5dde-4ec9-9c9b-d6e4a072fca7 None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:36:11.997869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1d8e6e75-5dde-4ec9-9c9b-d6e4a072fca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:12.009670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1139/2289] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:36:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:36:12.020855 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db27c1-0b13-4596-ae03-e05ebba56f0f None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:36:12.035156 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db27c1-0b13-4596-ae03-e05ebba56f0f None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:36:12.039821 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9043748-4841-4261-90be-7daab45667c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:36:12.040424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9043748-4841-4261-90be-7daab45667c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:12.040724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9043748-4841-4261-90be-7daab45667c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:12.041000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9043748-4841-4261-90be-7daab45667c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:12.052169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9043748-4841-4261-90be-7daab45667c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:36:12.053653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9043748-4841-4261-90be-7daab45667c9 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:36:12.065650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1151/2290] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:36:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:36:19.050590 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c6921-afa5-4835-b892-719d904b3a25 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:19.051560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c6921-afa5-4835-b892-719d904b3a25 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:19.051839 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c6921-afa5-4835-b892-719d904b3a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:19.080548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-060c6921-afa5-4835-b892-719d904b3a25 None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:36:19.081835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-060c6921-afa5-4835-b892-719d904b3a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:19.089582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1140/2291] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:36:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:36:19.102973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9043748-4841-4261-90be-7daab45667c9 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:36:19.115888 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9043748-4841-4261-90be-7daab45667c9 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:36:19.128071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b73aa2-5ff3-467c-8722-3937c8b088c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:36:19.128723 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b73aa2-5ff3-467c-8722-3937c8b088c4 service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:19.129007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b73aa2-5ff3-467c-8722-3937c8b088c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:19.129280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b73aa2-5ff3-467c-8722-3937c8b088c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:19.139608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b73aa2-5ff3-467c-8722-3937c8b088c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:36:19.141395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b73aa2-5ff3-467c-8722-3937c8b088c4 service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:36:19.154831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1152/2292] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:36:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:36:29.073300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=75 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:36:29.147444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=41 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:36:40.056478 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d5d25-624f-46d2-b561-e8bede32190f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:40.057109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d5d25-624f-46d2-b561-e8bede32190f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:40.057109 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d5d25-624f-46d2-b561-e8bede32190f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:40.079394 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-728d5d25-624f-46d2-b561-e8bede32190f None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:36:40.080628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-728d5d25-624f-46d2-b561-e8bede32190f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:40.088755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1141/2293] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 24 12:36:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:36:40.103410 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b73aa2-5ff3-467c-8722-3937c8b088c4 service swift] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:36:40.115066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b73aa2-5ff3-467c-8722-3937c8b088c4 service swift] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:36:40.122106 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22ca1f-48af-47c4-a552-92bce390e739 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:36:40.122704 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22ca1f-48af-47c4-a552-92bce390e739 service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:40.122982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22ca1f-48af-47c4-a552-92bce390e739 service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:40.123496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22ca1f-48af-47c4-a552-92bce390e739 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:40.132270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22ca1f-48af-47c4-a552-92bce390e739 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:36:40.134006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22ca1f-48af-47c4-a552-92bce390e739 service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:36:40.145009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1153/2294] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:36:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:36:50.073854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:36:50.140521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=6 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:36:50.694901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58835d4-f69b-4dab-9a55-1b79a08881fa None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:50.695414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58835d4-f69b-4dab-9a55-1b79a08881fa None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:50.695845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58835d4-f69b-4dab-9a55-1b79a08881fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:50.718025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a58835d4-f69b-4dab-9a55-1b79a08881fa None None] MFA Rules not processed for user `35d65aef527d421c979e400774fc3fee`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:36:50.719322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a58835d4-f69b-4dab-9a55-1b79a08881fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:50.727041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1142/2295] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:36:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:37:00.711089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:05.351964 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc53419-15cc-40c6-8ba2-2281bfc19718 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:05.352473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc53419-15cc-40c6-8ba2-2281bfc19718 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:05.352747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc53419-15cc-40c6-8ba2-2281bfc19718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:05.372113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1dc53419-15cc-40c6-8ba2-2281bfc19718 None None] MFA Rules not processed for user `b4a617fc13534a89ad50a5fdc7bf769a`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:05.373366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1dc53419-15cc-40c6-8ba2-2281bfc19718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:05.433435 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1154/2296] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:37:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:11.304441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373e2f8-a419-4f7d-96ee-c9fa0a4fc10b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:11.305006 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373e2f8-a419-4f7d-96ee-c9fa0a4fc10b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:11.305105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373e2f8-a419-4f7d-96ee-c9fa0a4fc10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:11.324513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4373e2f8-a419-4f7d-96ee-c9fa0a4fc10b None None] MFA Rules not processed for user `e34cbcf69b4b4828a3bca077fc8d0694`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:11.325758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4373e2f8-a419-4f7d-96ee-c9fa0a4fc10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:11.399411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1143/2297] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:37:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:15.425559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=25 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:21.385414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=25 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:37.056832 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715011a-96b0-4aec-8f41-9dd7f4950c90 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:37.057688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715011a-96b0-4aec-8f41-9dd7f4950c90 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:37.057688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715011a-96b0-4aec-8f41-9dd7f4950c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:37.080031 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8715011a-96b0-4aec-8f41-9dd7f4950c90 None None] MFA Rules not processed for user `fd39206c155f4e1cb9f7f90fb5a10556`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:37.081593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8715011a-96b0-4aec-8f41-9dd7f4950c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:37.148484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1155/2298] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:37:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:38.943700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373e2f8-a419-4f7d-96ee-c9fa0a4fc10b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:38.954617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373e2f8-a419-4f7d-96ee-c9fa0a4fc10b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:38.959694 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85907d-1c4d-42b4-9aba-609adebae281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:38.960284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85907d-1c4d-42b4-9aba-609adebae281 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:38.960560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85907d-1c4d-42b4-9aba-609adebae281 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:38.960856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85907d-1c4d-42b4-9aba-609adebae281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:38.962956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2b85907d-1c4d-42b4-9aba-609adebae281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:38.978202 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85907d-1c4d-42b4-9aba-609adebae281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:38.979649 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85907d-1c4d-42b4-9aba-609adebae281 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:38.991167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1144/2299] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:40.111030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715011a-96b0-4aec-8f41-9dd7f4950c90 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:40.121010 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715011a-96b0-4aec-8f41-9dd7f4950c90 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:40.124639 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751e3e1-ae00-447c-a5b4-4119ba66be3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:40.125258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751e3e1-ae00-447c-a5b4-4119ba66be3b service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:40.125527 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751e3e1-ae00-447c-a5b4-4119ba66be3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:40.125785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751e3e1-ae00-447c-a5b4-4119ba66be3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:40.127890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8751e3e1-ae00-447c-a5b4-4119ba66be3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:40.142735 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751e3e1-ae00-447c-a5b4-4119ba66be3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:40.144462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751e3e1-ae00-447c-a5b4-4119ba66be3b service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:40.155554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1156/2300] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:48.888855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85907d-1c4d-42b4-9aba-609adebae281 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:48.898209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85907d-1c4d-42b4-9aba-609adebae281 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:48.902026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e25a7c-1799-46a7-9a6e-8ff038cf13c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:48.902614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e25a7c-1799-46a7-9a6e-8ff038cf13c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:48.902887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e25a7c-1799-46a7-9a6e-8ff038cf13c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:48.903164 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e25a7c-1799-46a7-9a6e-8ff038cf13c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:48.905297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a8e25a7c-1799-46a7-9a6e-8ff038cf13c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:48.966813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e25a7c-1799-46a7-9a6e-8ff038cf13c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:48.968266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e25a7c-1799-46a7-9a6e-8ff038cf13c9 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:48.978360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1145/2301] 10.222.0.27 () {60 vars in 1396 bytes} [Wed Aug 24 12:37:48 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:50.150637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=31 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:55.138645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375137a-d5c6-4d0f-82ae-d53b1942309c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:55.139250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375137a-d5c6-4d0f-82ae-d53b1942309c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:55.139250 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375137a-d5c6-4d0f-82ae-d53b1942309c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:55.159855 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b375137a-d5c6-4d0f-82ae-d53b1942309c None None] MFA Rules not processed for user `85d8138d44bc40dd8f63ee4916ffbce9`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:55.162943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b375137a-d5c6-4d0f-82ae-d53b1942309c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:55.189613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1157/2302] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:58.973595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=34 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:00.862306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e25a7c-1799-46a7-9a6e-8ff038cf13c9 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:00.869915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a8e25a7c-1799-46a7-9a6e-8ff038cf13c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:00.890933 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e25a7c-1799-46a7-9a6e-8ff038cf13c9 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:00.897395 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702be296-605a-4f18-869b-74f9161e48b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:00.898345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702be296-605a-4f18-869b-74f9161e48b0 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:00.898835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702be296-605a-4f18-869b-74f9161e48b0 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:00.899779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702be296-605a-4f18-869b-74f9161e48b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:00.903193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-702be296-605a-4f18-869b-74f9161e48b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:00.931650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375137a-d5c6-4d0f-82ae-d53b1942309c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:00.942351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702be296-605a-4f18-869b-74f9161e48b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:00.943999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702be296-605a-4f18-869b-74f9161e48b0 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:00.954886 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375137a-d5c6-4d0f-82ae-d53b1942309c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:00.956205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1146/2303] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:00.961120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26dcea-fe50-415e-8164-17d92c388df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:00.962008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26dcea-fe50-415e-8164-17d92c388df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:00.962285 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26dcea-fe50-415e-8164-17d92c388df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:00.962555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26dcea-fe50-415e-8164-17d92c388df3 admin admin] PATH_INFO: `/v3/users/35d65aef527d421c979e400774fc3fee` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:00.963987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26dcea-fe50-415e-8164-17d92c388df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35d65aef527d421c979e400774fc3fee)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:00.966144 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26dcea-fe50-415e-8164-17d92c388df3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:01.037240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1f26dcea-fe50-415e-8164-17d92c388df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35d65aef527d421c979e400774fc3fee'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:01.037887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1f26dcea-fe50-415e-8164-17d92c388df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35d65aef527d421c979e400774fc3fee'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:01.038485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1f26dcea-fe50-415e-8164-17d92c388df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35d65aef527d421c979e400774fc3fee'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:01.050873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1f26dcea-fe50-415e-8164-17d92c388df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35d65aef527d421c979e400774fc3fee'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:01.060002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1f26dcea-fe50-415e-8164-17d92c388df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35d65aef527d421c979e400774fc3fee'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:01.075460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1158/2304] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:00 2022] DELETE /identity/v3/users/35d65aef527d421c979e400774fc3fee => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:01.256233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702be296-605a-4f18-869b-74f9161e48b0 service placement] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:01.266896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702be296-605a-4f18-869b-74f9161e48b0 service placement] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:01.326863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:01.327501 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:01.327777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:01.328048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf admin admin] PATH_INFO: `/v3/projects/e3e2e4f6da374afba391fec0fa0bb3fb` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:01.329313 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e2e4f6da374afba391fec0fa0bb3fb)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:01.331396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:01.383994 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e2e4f6da374afba391fec0fa0bb3fb', 'request_id': 'req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:01.393847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e2e4f6da374afba391fec0fa0bb3fb', 'request_id': 'req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:01.400581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e2e4f6da374afba391fec0fa0bb3fb', 'request_id': 'req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:01.412677 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf admin admin] The token cache is being invalidate because project e3e2e4f6da374afba391fec0fa0bb3fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:01.413005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:01.413742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1f3fd92d-a09f-4fc4-b442-34fb4af07abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:01.415351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1147/2305] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:01 2022] DELETE /identity/v3/projects/e3e2e4f6da374afba391fec0fa0bb3fb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:01.421770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acb2a4-8704-4fd2-b388-fc3679d845b0 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:01.422061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acb2a4-8704-4fd2-b388-fc3679d845b0 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:01.422336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acb2a4-8704-4fd2-b388-fc3679d845b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:01.444621 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-02acb2a4-8704-4fd2-b388-fc3679d845b0 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:01.446539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-02acb2a4-8704-4fd2-b388-fc3679d845b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:01.511000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1159/2306] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:38:01.584203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bb8aa-db2e-4cc9-ba61-c8431bb61f55 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:01.584729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bb8aa-db2e-4cc9-ba61-c8431bb61f55 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:01.585183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bb8aa-db2e-4cc9-ba61-c8431bb61f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:01.606778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-dc3bb8aa-db2e-4cc9-ba61-c8431bb61f55 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:01.608091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dc3bb8aa-db2e-4cc9-ba61-c8431bb61f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:01.616840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1148/2307] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:01.621056 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02acb2a4-8704-4fd2-b388-fc3679d845b0 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:01.633243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02acb2a4-8704-4fd2-b388-fc3679d845b0 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:01.638128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bdb44-ceab-4a43-be91-a2cb307a74a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:01.638734 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bdb44-ceab-4a43-be91-a2cb307a74a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:01.639011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bdb44-ceab-4a43-be91-a2cb307a74a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:01.639280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bdb44-ceab-4a43-be91-a2cb307a74a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:01.639927 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bdb44-ceab-4a43-be91-a2cb307a74a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:38:01.640228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bdb44-ceab-4a43-be91-a2cb307a74a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:01.642116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bdb44-ceab-4a43-be91-a2cb307a74a7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:01.655013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1160/2308] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:38:01.659684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bb8aa-db2e-4cc9-ba61-c8431bb61f55 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:01.669510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bb8aa-db2e-4cc9-ba61-c8431bb61f55 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:01.675302 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda39ddc-43e7-4cf7-95b6-b3b4e916f05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:01.675916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda39ddc-43e7-4cf7-95b6-b3b4e916f05f admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:01.676192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda39ddc-43e7-4cf7-95b6-b3b4e916f05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:01.676492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda39ddc-43e7-4cf7-95b6-b3b4e916f05f admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:01.677159 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda39ddc-43e7-4cf7-95b6-b3b4e916f05f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:01.679169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda39ddc-43e7-4cf7-95b6-b3b4e916f05f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:01.712749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1149/2309] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:01 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:01.713895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bdb44-ceab-4a43-be91-a2cb307a74a7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:01.724575 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bdb44-ceab-4a43-be91-a2cb307a74a7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:01.797696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:01.809725 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1cdf1-7818-453d-b72c-97bf752bbbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:01.810577 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1cdf1-7818-453d-b72c-97bf752bbbd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:01.812970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1cdf1-7818-453d-b72c-97bf752bbbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:01.813508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1cdf1-7818-453d-b72c-97bf752bbbd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:01.814384 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1cdf1-7818-453d-b72c-97bf752bbbd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:01.816462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1cdf1-7818-453d-b72c-97bf752bbbd9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:01.843369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1161/2310] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:01 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:38:01.847603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda39ddc-43e7-4cf7-95b6-b3b4e916f05f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:01.859472 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda39ddc-43e7-4cf7-95b6-b3b4e916f05f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:01.865268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914709a-802e-4114-a3a4-7e21b65c1534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:01.866001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914709a-802e-4114-a3a4-7e21b65c1534 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:01.866344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914709a-802e-4114-a3a4-7e21b65c1534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:01.866681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914709a-802e-4114-a3a4-7e21b65c1534 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:01.867404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914709a-802e-4114-a3a4-7e21b65c1534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:01.869211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914709a-802e-4114-a3a4-7e21b65c1534 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:01.883532 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1150/2311] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:38:01.888023 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1cdf1-7818-453d-b72c-97bf752bbbd9 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:01.900123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1cdf1-7818-453d-b72c-97bf752bbbd9 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:01.905111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36d2c8-874f-4ff4-a488-3a0956b1f63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:01.905719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36d2c8-874f-4ff4-a488-3a0956b1f63c admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:01.905992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36d2c8-874f-4ff4-a488-3a0956b1f63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:01.906260 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36d2c8-874f-4ff4-a488-3a0956b1f63c admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:01.906921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36d2c8-874f-4ff4-a488-3a0956b1f63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:01.909020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36d2c8-874f-4ff4-a488-3a0956b1f63c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:01.925339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1162/2312] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:01.927800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914709a-802e-4114-a3a4-7e21b65c1534 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:01.939022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914709a-802e-4114-a3a4-7e21b65c1534 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:01.944758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a995cb-820f-4494-98ae-874207b3c010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:01.945609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a995cb-820f-4494-98ae-874207b3c010 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:01.945890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a995cb-820f-4494-98ae-874207b3c010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:01.946226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a995cb-820f-4494-98ae-874207b3c010 admin admin] PATH_INFO: `/v3/projects/e112083132a9492089fafdae9e34f4ff/users/ae1cd97c879141e2b4c1527b5964d054/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:01.963847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a995cb-820f-4494-98ae-874207b3c010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e112083132a9492089fafdae9e34f4ff, user_id=ae1cd97c879141e2b4c1527b5964d054, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:01.966672 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a995cb-820f-4494-98ae-874207b3c010 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:01.989957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1151/2313] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:01 2022] PUT /identity/v3/projects/e112083132a9492089fafdae9e34f4ff/users/ae1cd97c879141e2b4c1527b5964d054/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:01.996268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18f300-6b4b-42b6-a764-308847b83f6b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:01.996851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18f300-6b4b-42b6-a764-308847b83f6b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:01.996851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18f300-6b4b-42b6-a764-308847b83f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:02.016266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-eb18f300-6b4b-42b6-a764-308847b83f6b None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:02.018101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-eb18f300-6b4b-42b6-a764-308847b83f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:02.079600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1163/2314] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:02.089796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a995cb-820f-4494-98ae-874207b3c010 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:02.091556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b0a995cb-820f-4494-98ae-874207b3c010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:02.106905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:02.164802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a995cb-820f-4494-98ae-874207b3c010 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:02.171082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba0f70-8201-437b-a4c5-8a3a49af9df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:02.171717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba0f70-8201-437b-a4c5-8a3a49af9df9 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:02.172000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba0f70-8201-437b-a4c5-8a3a49af9df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:02.172305 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba0f70-8201-437b-a4c5-8a3a49af9df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:02.183561 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba0f70-8201-437b-a4c5-8a3a49af9df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:02.185089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba0f70-8201-437b-a4c5-8a3a49af9df9 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:02.198837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1152/2315] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:02.578679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce1493-615a-4b37-8980-4555bde97934 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:02.579445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce1493-615a-4b37-8980-4555bde97934 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:02.579978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce1493-615a-4b37-8980-4555bde97934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:02.602203 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-96ce1493-615a-4b37-8980-4555bde97934 None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:02.603796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-96ce1493-615a-4b37-8980-4555bde97934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:02.669534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1164/2316] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 24 12:38:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:02.720670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba0f70-8201-437b-a4c5-8a3a49af9df9 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:02.722678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eeba0f70-8201-437b-a4c5-8a3a49af9df9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:02.802724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba0f70-8201-437b-a4c5-8a3a49af9df9 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:02.807847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f65ed-d948-47b0-afef-eb4417c214a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:02.808530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f65ed-d948-47b0-afef-eb4417c214a2 service swift] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:02.808963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f65ed-d948-47b0-afef-eb4417c214a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:02.809243 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f65ed-d948-47b0-afef-eb4417c214a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:02.818887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f65ed-d948-47b0-afef-eb4417c214a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:02.820673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f65ed-d948-47b0-afef-eb4417c214a2 service swift] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:02.829305 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1153/2317] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:38:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:38:06.474585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce1493-615a-4b37-8980-4555bde97934 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:06.476365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-96ce1493-615a-4b37-8980-4555bde97934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:06.554453 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce1493-615a-4b37-8980-4555bde97934 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:06.560172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf9f57-b9e4-44ad-87f7-321331ebcf2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:06.561123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf9f57-b9e4-44ad-87f7-321331ebcf2e service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:06.562084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf9f57-b9e4-44ad-87f7-321331ebcf2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:06.562461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf9f57-b9e4-44ad-87f7-321331ebcf2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:06.564560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9faf9f57-b9e4-44ad-87f7-321331ebcf2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:06.613578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:06.654471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf9f57-b9e4-44ad-87f7-321331ebcf2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:06.655921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf9f57-b9e4-44ad-87f7-321331ebcf2e service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:06.667117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1165/2318] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:06 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:08.680920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f26374-90b9-494d-88c1-db4cb57a8961 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:08.681853 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f26374-90b9-494d-88c1-db4cb57a8961 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:08.682307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f26374-90b9-494d-88c1-db4cb57a8961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:08.720282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b3f26374-90b9-494d-88c1-db4cb57a8961 None None] MFA Rules not processed for user `ae1cd97c879141e2b4c1527b5964d054`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:08.722308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b3f26374-90b9-494d-88c1-db4cb57a8961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:08.827543 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1154/2319] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:08.833151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf9f57-b9e4-44ad-87f7-321331ebcf2e service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:08.852876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf9f57-b9e4-44ad-87f7-321331ebcf2e service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:08.864768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b6808-2acf-48af-a0fc-70f1b4cc4808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:08.865646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b6808-2acf-48af-a0fc-70f1b4cc4808 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:08.866181 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b6808-2acf-48af-a0fc-70f1b4cc4808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:08.866664 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b6808-2acf-48af-a0fc-70f1b4cc4808 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:08.867582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b6808-2acf-48af-a0fc-70f1b4cc4808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:08.869788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b6808-2acf-48af-a0fc-70f1b4cc4808 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:08.903440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1166/2320] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:08 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:08.907713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f26374-90b9-494d-88c1-db4cb57a8961 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:08.919564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f26374-90b9-494d-88c1-db4cb57a8961 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:08.956929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:08.989514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501b359-4b0e-4796-9167-e3f63f3a0ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:08.990210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501b359-4b0e-4796-9167-e3f63f3a0ffa admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:08.990592 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501b359-4b0e-4796-9167-e3f63f3a0ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:08.990921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501b359-4b0e-4796-9167-e3f63f3a0ffa admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:08.991632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501b359-4b0e-4796-9167-e3f63f3a0ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:08.993668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501b359-4b0e-4796-9167-e3f63f3a0ffa admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:09.019045 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1155/2321] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:08 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:09.023552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b6808-2acf-48af-a0fc-70f1b4cc4808 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:09.035011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b6808-2acf-48af-a0fc-70f1b4cc4808 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:09.039689 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be166f-c3dc-43cc-a151-4c0634d5b084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:09.040293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be166f-c3dc-43cc-a151-4c0634d5b084 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:09.040570 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be166f-c3dc-43cc-a151-4c0634d5b084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:09.040870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be166f-c3dc-43cc-a151-4c0634d5b084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:09.041552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be166f-c3dc-43cc-a151-4c0634d5b084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:09.043189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be166f-c3dc-43cc-a151-4c0634d5b084 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:09.056100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1167/2322] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:09.060682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501b359-4b0e-4796-9167-e3f63f3a0ffa admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:09.071348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501b359-4b0e-4796-9167-e3f63f3a0ffa admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:09.076304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cbba0-285a-4ac0-822d-7342503cd41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:09.076995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cbba0-285a-4ac0-822d-7342503cd41d admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:09.077320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cbba0-285a-4ac0-822d-7342503cd41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:09.077594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cbba0-285a-4ac0-822d-7342503cd41d admin admin] PATH_INFO: `/v3/projects/63e3d5bb531543c4afae2a644b2c6870/users/852082d3b62f49b48881b81bf3ce1e4b/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:09.096262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255cbba0-285a-4ac0-822d-7342503cd41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e3d5bb531543c4afae2a644b2c6870, user_id=852082d3b62f49b48881b81bf3ce1e4b, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:09.099474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255cbba0-285a-4ac0-822d-7342503cd41d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:09.117844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1156/2323] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:09 2022] PUT /identity/v3/projects/63e3d5bb531543c4afae2a644b2c6870/users/852082d3b62f49b48881b81bf3ce1e4b/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:09.123062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be166f-c3dc-43cc-a151-4c0634d5b084 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:09.135830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be166f-c3dc-43cc-a151-4c0634d5b084 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:09.230520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987f844-c911-435d-b5b3-e362c652fd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:09.231154 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987f844-c911-435d-b5b3-e362c652fd4a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:09.231429 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987f844-c911-435d-b5b3-e362c652fd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:09.231694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987f844-c911-435d-b5b3-e362c652fd4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:09.232674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987f844-c911-435d-b5b3-e362c652fd4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:09.234001 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987f844-c911-435d-b5b3-e362c652fd4a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:09.245836 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1168/2324] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:09.249969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cbba0-285a-4ac0-822d-7342503cd41d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:09.269362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cbba0-285a-4ac0-822d-7342503cd41d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:09.277389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d97cc-9ecc-469f-9e43-a89fc226bac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:09.277991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d97cc-9ecc-469f-9e43-a89fc226bac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:09.278272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d97cc-9ecc-469f-9e43-a89fc226bac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:09.278545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d97cc-9ecc-469f-9e43-a89fc226bac7 admin admin] PATH_INFO: `/v3/projects/63e3d5bb531543c4afae2a644b2c6870/users/852082d3b62f49b48881b81bf3ce1e4b/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:09.285449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d97cc-9ecc-469f-9e43-a89fc226bac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e3d5bb531543c4afae2a644b2c6870, user_id=852082d3b62f49b48881b81bf3ce1e4b, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:09.288619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d97cc-9ecc-469f-9e43-a89fc226bac7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:09.301204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1157/2325] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:09 2022] PUT /identity/v3/projects/63e3d5bb531543c4afae2a644b2c6870/users/852082d3b62f49b48881b81bf3ce1e4b/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:38:10.788880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987f844-c911-435d-b5b3-e362c652fd4a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:10.791298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2987f844-c911-435d-b5b3-e362c652fd4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:10.964127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987f844-c911-435d-b5b3-e362c652fd4a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:11.034130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:11.233034 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a43bdf-1b6e-4efb-9b2e-b317345e4a2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:11.234107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a43bdf-1b6e-4efb-9b2e-b317345e4a2f service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:11.234662 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a43bdf-1b6e-4efb-9b2e-b317345e4a2f service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:11.235139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a43bdf-1b6e-4efb-9b2e-b317345e4a2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:11.238847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-11a43bdf-1b6e-4efb-9b2e-b317345e4a2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:11.393790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a43bdf-1b6e-4efb-9b2e-b317345e4a2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:11.401146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a43bdf-1b6e-4efb-9b2e-b317345e4a2f service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:11.429748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1169/2326] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:12.386090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d97cc-9ecc-469f-9e43-a89fc226bac7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:12.388088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f74d97cc-9ecc-469f-9e43-a89fc226bac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:12.533619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d97cc-9ecc-469f-9e43-a89fc226bac7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:12.545236 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbbf8c5-5c90-4d94-ac99-2c793354782f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:12.547265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbf8c5-5c90-4d94-ac99-2c793354782f service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:12.548341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbf8c5-5c90-4d94-ac99-2c793354782f service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:12.550304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbbf8c5-5c90-4d94-ac99-2c793354782f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:12.553492 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cfbbf8c5-5c90-4d94-ac99-2c793354782f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:12.572421 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a43bdf-1b6e-4efb-9b2e-b317345e4a2f service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:12.572421 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-11a43bdf-1b6e-4efb-9b2e-b317345e4a2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:12.591967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbbf8c5-5c90-4d94-ac99-2c793354782f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:12.594010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbbf8c5-5c90-4d94-ac99-2c793354782f service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:12.613836 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1158/2327] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:12.707378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a43bdf-1b6e-4efb-9b2e-b317345e4a2f service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:12.718223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:12.719331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:12.721444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:12.722231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] PATH_INFO: `/v3/users/3a145fea1ee64ab18b93f3bf47c3fb65` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:12.724216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a145fea1ee64ab18b93f3bf47c3fb65)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:12.726463 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:12.796996 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a145fea1ee64ab18b93f3bf47c3fb65'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:12.797814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a145fea1ee64ab18b93f3bf47c3fb65'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:12.799645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a145fea1ee64ab18b93f3bf47c3fb65'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:12.829724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a145fea1ee64ab18b93f3bf47c3fb65'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:12.845765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a145fea1ee64ab18b93f3bf47c3fb65'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:12.847807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:12.864172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1170/2328] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:12 2022] DELETE /identity/v3/users/3a145fea1ee64ab18b93f3bf47c3fb65 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:38:13.128974 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbbf8c5-5c90-4d94-ac99-2c793354782f service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:13.145299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbbf8c5-5c90-4d94-ac99-2c793354782f service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:13.255755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:13.256520 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:13.256886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:13.257225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] PATH_INFO: `/v3/projects/5dc78d8e41d843a2b085f8300a285a0e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:13.265465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc78d8e41d843a2b085f8300a285a0e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:13.280925 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:13.336190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:13.377783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc78d8e41d843a2b085f8300a285a0e', 'request_id': 'req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:13.392964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc78d8e41d843a2b085f8300a285a0e', 'request_id': 'req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:13.404399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc78d8e41d843a2b085f8300a285a0e', 'request_id': 'req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:13.434277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] The token cache is being invalidate because project 5dc78d8e41d843a2b085f8300a285a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:13.434277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:13.437409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:13.444487 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1159/2329] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:13 2022] DELETE /identity/v3/projects/5dc78d8e41d843a2b085f8300a285a0e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:13.445929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:13.451364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:13.478492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ad9e9-f8e0-4ec9-af2f-90694c92b47e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:13.569835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1cce0-3711-48fd-b9fa-d27079089918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:13.570468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1cce0-3711-48fd-b9fa-d27079089918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:13.570918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1cce0-3711-48fd-b9fa-d27079089918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:13.571323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1cce0-3711-48fd-b9fa-d27079089918 admin admin] PATH_INFO: `/v3/users/b33d94b97d51408286b7916e8b2a2fa7` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:13.572639 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1cce0-3711-48fd-b9fa-d27079089918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b33d94b97d51408286b7916e8b2a2fa7)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:13.574921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1cce0-3711-48fd-b9fa-d27079089918 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:13.654040 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b6d1cce0-3711-48fd-b9fa-d27079089918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b33d94b97d51408286b7916e8b2a2fa7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:13.655678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b6d1cce0-3711-48fd-b9fa-d27079089918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b33d94b97d51408286b7916e8b2a2fa7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:13.657354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b6d1cce0-3711-48fd-b9fa-d27079089918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b33d94b97d51408286b7916e8b2a2fa7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:13.702583 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b6d1cce0-3711-48fd-b9fa-d27079089918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b33d94b97d51408286b7916e8b2a2fa7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:13.735118 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b6d1cce0-3711-48fd-b9fa-d27079089918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b33d94b97d51408286b7916e8b2a2fa7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:13.825146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1171/2330] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:13 2022] DELETE /identity/v3/users/b33d94b97d51408286b7916e8b2a2fa7 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:14.575620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:14.577803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:14.609180 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0e5ef-4b19-4c40-9d6b-480da24ac84a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:14.755253 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:14.756130 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:14.756595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:14.757060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] PATH_INFO: `/v3/projects/57051d7f1c4f45ae818325105ec0d5be` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:14.758856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57051d7f1c4f45ae818325105ec0d5be)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:14.765516 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:14.861785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57051d7f1c4f45ae818325105ec0d5be', 'request_id': 'req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:14.880440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57051d7f1c4f45ae818325105ec0d5be', 'request_id': 'req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:14.889485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57051d7f1c4f45ae818325105ec0d5be', 'request_id': 'req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:14.923399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] The token cache is being invalidate because project 57051d7f1c4f45ae818325105ec0d5be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:14.924166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:14.925001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:14.936999 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1160/2331] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:14 2022] DELETE /identity/v3/projects/57051d7f1c4f45ae818325105ec0d5be => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:17.612301 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2da4c-56cf-4475-b676-9b5dcb2a3f7c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:17.612886 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2da4c-56cf-4475-b676-9b5dcb2a3f7c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:17.612886 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2da4c-56cf-4475-b676-9b5dcb2a3f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:17.644175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-81f2da4c-56cf-4475-b676-9b5dcb2a3f7c None None] MFA Rules not processed for user `852082d3b62f49b48881b81bf3ce1e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:17.645443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-81f2da4c-56cf-4475-b676-9b5dcb2a3f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:17.711758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:17.732953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1172/2332] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:17.742294 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:17.743837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:17.772623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2da4c-56cf-4475-b676-9b5dcb2a3f7c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:17.774327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-81f2da4c-56cf-4475-b676-9b5dcb2a3f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:17.810719 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:17.839318 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07b4e4-f7cd-4819-9fe0-e803b5025f98 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:17.845320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce978f-7806-443a-ab06-2c67eda17dea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:17.847367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce978f-7806-443a-ab06-2c67eda17dea service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:17.847647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce978f-7806-443a-ab06-2c67eda17dea service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:17.847915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce978f-7806-443a-ab06-2c67eda17dea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:17.849876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-00ce978f-7806-443a-ab06-2c67eda17dea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:17.867462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2da4c-56cf-4475-b676-9b5dcb2a3f7c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:17.873731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecdca2-0726-4cb1-aeb9-fc40691368b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:17.874368 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecdca2-0726-4cb1-aeb9-fc40691368b3 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:17.874667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecdca2-0726-4cb1-aeb9-fc40691368b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:17.874939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecdca2-0726-4cb1-aeb9-fc40691368b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:17.877500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d6ecdca2-0726-4cb1-aeb9-fc40691368b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:17.899545 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecdca2-0726-4cb1-aeb9-fc40691368b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:17.902050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecdca2-0726-4cb1-aeb9-fc40691368b3 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:17.926263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce978f-7806-443a-ab06-2c67eda17dea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:17.928360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce978f-7806-443a-ab06-2c67eda17dea service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:17.929663 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1173/2333] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:17.955011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1161/2334] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:23.319991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecdca2-0726-4cb1-aeb9-fc40691368b3 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:23.333436 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecdca2-0726-4cb1-aeb9-fc40691368b3 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:23.340531 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd7322-12c6-4049-9ca5-e4268f821fc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:23.341410 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd7322-12c6-4049-9ca5-e4268f821fc1 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:23.341847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd7322-12c6-4049-9ca5-e4268f821fc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:23.342261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd7322-12c6-4049-9ca5-e4268f821fc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:23.345240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c5fd7322-12c6-4049-9ca5-e4268f821fc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:23.359246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd7322-12c6-4049-9ca5-e4268f821fc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:23.361578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd7322-12c6-4049-9ca5-e4268f821fc1 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:23.373569 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1174/2335] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:27.935185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=33 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:30.538685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce978f-7806-443a-ab06-2c67eda17dea service nova] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:30.540344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-00ce978f-7806-443a-ab06-2c67eda17dea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:30.638669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce978f-7806-443a-ab06-2c67eda17dea service nova] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:30.643998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f6913-0d85-4011-b678-9a987c4a4407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:30.644604 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f6913-0d85-4011-b678-9a987c4a4407 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:30.644904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f6913-0d85-4011-b678-9a987c4a4407 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:30.645172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f6913-0d85-4011-b678-9a987c4a4407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:30.647040 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-281f6913-0d85-4011-b678-9a987c4a4407 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:30.715296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f6913-0d85-4011-b678-9a987c4a4407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:30.717433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f6913-0d85-4011-b678-9a987c4a4407 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:30.729969 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1162/2336] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:33.368989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=43 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:34.061850 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd7322-12c6-4049-9ca5-e4268f821fc1 service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:34.064955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c5fd7322-12c6-4049-9ca5-e4268f821fc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:34.089109 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd7322-12c6-4049-9ca5-e4268f821fc1 service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:34.094573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623906e2-53d2-4742-9371-82a1279958ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:34.095531 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623906e2-53d2-4742-9371-82a1279958ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:34.095697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623906e2-53d2-4742-9371-82a1279958ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:34.096053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623906e2-53d2-4742-9371-82a1279958ef admin admin] PATH_INFO: `/v3/users/e34cbcf69b4b4828a3bca077fc8d0694` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:34.097306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623906e2-53d2-4742-9371-82a1279958ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e34cbcf69b4b4828a3bca077fc8d0694)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:34.099463 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623906e2-53d2-4742-9371-82a1279958ef admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:34.155745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-623906e2-53d2-4742-9371-82a1279958ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e34cbcf69b4b4828a3bca077fc8d0694'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.156569 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-623906e2-53d2-4742-9371-82a1279958ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e34cbcf69b4b4828a3bca077fc8d0694'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.157293 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-623906e2-53d2-4742-9371-82a1279958ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e34cbcf69b4b4828a3bca077fc8d0694'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.174230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-623906e2-53d2-4742-9371-82a1279958ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e34cbcf69b4b4828a3bca077fc8d0694'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.185686 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-623906e2-53d2-4742-9371-82a1279958ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e34cbcf69b4b4828a3bca077fc8d0694'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.204474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1175/2337] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:34 2022] DELETE /identity/v3/users/e34cbcf69b4b4828a3bca077fc8d0694 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:34.470216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f6913-0d85-4011-b678-9a987c4a4407 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:34.486883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f6913-0d85-4011-b678-9a987c4a4407 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:34.571954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:34.572585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:34.572897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:34.573167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd admin admin] PATH_INFO: `/v3/projects/3276a7bc6bb84ac7953d2d46c4298a23` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:34.574360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3276a7bc6bb84ac7953d2d46c4298a23)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:34.576278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:34.634553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3276a7bc6bb84ac7953d2d46c4298a23', 'request_id': 'req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.639479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:34.646066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3276a7bc6bb84ac7953d2d46c4298a23', 'request_id': 'req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.657068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3276a7bc6bb84ac7953d2d46c4298a23', 'request_id': 'req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.670741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd admin admin] The token cache is being invalidate because project 3276a7bc6bb84ac7953d2d46c4298a23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:34.671058 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.671804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ea09a4b4-0281-431b-ad0a-594d0e50d7bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.673760 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1163/2338] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:34 2022] DELETE /identity/v3/projects/3276a7bc6bb84ac7953d2d46c4298a23 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:34.679365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924b3b3-9a76-4162-a8a9-cac3175503ea None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:34.679852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924b3b3-9a76-4162-a8a9-cac3175503ea None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:34.680383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924b3b3-9a76-4162-a8a9-cac3175503ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:34.703222 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b924b3b3-9a76-4162-a8a9-cac3175503ea None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:34.704417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b924b3b3-9a76-4162-a8a9-cac3175503ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:34.770529 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1176/2339] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:34.778626 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f109ef2-b4e0-4f7b-b838-d2e1ddcf9820 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:34.778967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f109ef2-b4e0-4f7b-b838-d2e1ddcf9820 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:34.779309 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f109ef2-b4e0-4f7b-b838-d2e1ddcf9820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:34.799965 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2f109ef2-b4e0-4f7b-b838-d2e1ddcf9820 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:34.801344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2f109ef2-b4e0-4f7b-b838-d2e1ddcf9820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:34.810645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1164/2340] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:34.814839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924b3b3-9a76-4162-a8a9-cac3175503ea None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:34.825637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924b3b3-9a76-4162-a8a9-cac3175503ea None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:34.831500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df4aa5-370f-4c5c-a52f-e3f7e133a874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:34.832160 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4aa5-370f-4c5c-a52f-e3f7e133a874 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:34.832470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4aa5-370f-4c5c-a52f-e3f7e133a874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:34.832774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4aa5-370f-4c5c-a52f-e3f7e133a874 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:34.833518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df4aa5-370f-4c5c-a52f-e3f7e133a874 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:38:34.833824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df4aa5-370f-4c5c-a52f-e3f7e133a874 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:34.836185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df4aa5-370f-4c5c-a52f-e3f7e133a874 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:34.850778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1177/2341] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:38:34.855768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f109ef2-b4e0-4f7b-b838-d2e1ddcf9820 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:34.866415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f109ef2-b4e0-4f7b-b838-d2e1ddcf9820 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:34.871485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f750b2d-fc06-4259-a211-7b1c6400ed74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:34.872199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f750b2d-fc06-4259-a211-7b1c6400ed74 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:34.872539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f750b2d-fc06-4259-a211-7b1c6400ed74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:34.873016 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f750b2d-fc06-4259-a211-7b1c6400ed74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:34.873657 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f750b2d-fc06-4259-a211-7b1c6400ed74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:34.875803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f750b2d-fc06-4259-a211-7b1c6400ed74 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:34.910943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1165/2342] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:34 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:34.914350 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df4aa5-370f-4c5c-a52f-e3f7e133a874 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:34.926659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df4aa5-370f-4c5c-a52f-e3f7e133a874 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:34.990445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d73f93-81dd-4722-91c1-51a7f9f79b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:34.991063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d73f93-81dd-4722-91c1-51a7f9f79b6a admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:34.991391 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d73f93-81dd-4722-91c1-51a7f9f79b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:34.991663 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d73f93-81dd-4722-91c1-51a7f9f79b6a admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:34.992289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d73f93-81dd-4722-91c1-51a7f9f79b6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:34.994282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d73f93-81dd-4722-91c1-51a7f9f79b6a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.002524 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:35.020715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1178/2343] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:34 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:35.023428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f750b2d-fc06-4259-a211-7b1c6400ed74 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.034371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f750b2d-fc06-4259-a211-7b1c6400ed74 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.039144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12926ffb-cd80-4483-b66f-8848163f8c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.039879 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12926ffb-cd80-4483-b66f-8848163f8c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.040157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12926ffb-cd80-4483-b66f-8848163f8c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.040537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12926ffb-cd80-4483-b66f-8848163f8c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.041304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12926ffb-cd80-4483-b66f-8848163f8c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.042915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12926ffb-cd80-4483-b66f-8848163f8c77 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.054220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1166/2344] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:35.058359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d73f93-81dd-4722-91c1-51a7f9f79b6a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.070013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d73f93-81dd-4722-91c1-51a7f9f79b6a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.074828 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff34b0-4c74-4a2b-9c9f-354477b26b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.075533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff34b0-4c74-4a2b-9c9f-354477b26b11 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.075904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff34b0-4c74-4a2b-9c9f-354477b26b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.076431 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff34b0-4c74-4a2b-9c9f-354477b26b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.076915 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff34b0-4c74-4a2b-9c9f-354477b26b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.078883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff34b0-4c74-4a2b-9c9f-354477b26b11 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.090161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1179/2345] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:35.094706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12926ffb-cd80-4483-b66f-8848163f8c77 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.105394 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12926ffb-cd80-4483-b66f-8848163f8c77 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.111498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87402567-8559-449c-bca2-52521b3d18c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.112147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87402567-8559-449c-bca2-52521b3d18c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.112426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87402567-8559-449c-bca2-52521b3d18c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.112728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87402567-8559-449c-bca2-52521b3d18c3 admin admin] PATH_INFO: `/v3/projects/5d67f75cc6904cc889ca0f678df98b9c/users/e9385e35bc7c462c87cf9e4092708bd9/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.129828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87402567-8559-449c-bca2-52521b3d18c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d67f75cc6904cc889ca0f678df98b9c, user_id=e9385e35bc7c462c87cf9e4092708bd9, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.132516 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87402567-8559-449c-bca2-52521b3d18c3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.148125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1167/2346] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:35 2022] PUT /identity/v3/projects/5d67f75cc6904cc889ca0f678df98b9c/users/e9385e35bc7c462c87cf9e4092708bd9/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:35.158993 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29081f39-191f-4357-80b3-89d83c74c77d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.159280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29081f39-191f-4357-80b3-89d83c74c77d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.159555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29081f39-191f-4357-80b3-89d83c74c77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.189762 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-29081f39-191f-4357-80b3-89d83c74c77d None None] MFA Rules not processed for user `e9385e35bc7c462c87cf9e4092708bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:35.191008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-29081f39-191f-4357-80b3-89d83c74c77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:35.259284 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1180/2347] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:35.271432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87402567-8559-449c-bca2-52521b3d18c3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.273432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-87402567-8559-449c-bca2-52521b3d18c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:35.276404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a259a34-984c-4990-a223-7ae459aaaf19 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.276743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a259a34-984c-4990-a223-7ae459aaaf19 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.277015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a259a34-984c-4990-a223-7ae459aaaf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.296166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-4a259a34-984c-4990-a223-7ae459aaaf19 None None] MFA Rules not processed for user `b4a617fc13534a89ad50a5fdc7bf769a`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:35.297376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4a259a34-984c-4990-a223-7ae459aaaf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:35.355465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87402567-8559-449c-bca2-52521b3d18c3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.359557 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1181/2348] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:35.365871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272424eb-6a80-43d5-a0f1-0d6ebe45a035 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.366486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272424eb-6a80-43d5-a0f1-0d6ebe45a035 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.366763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272424eb-6a80-43d5-a0f1-0d6ebe45a035 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.367034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272424eb-6a80-43d5-a0f1-0d6ebe45a035 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.375857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272424eb-6a80-43d5-a0f1-0d6ebe45a035 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.377886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272424eb-6a80-43d5-a0f1-0d6ebe45a035 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.390211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1168/2349] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:35.698211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a259a34-984c-4990-a223-7ae459aaaf19 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.699853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4a259a34-984c-4990-a223-7ae459aaaf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:35.772275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a259a34-984c-4990-a223-7ae459aaaf19 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.776308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d49f83-df5a-46d3-a33f-3bddbddc813b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.776961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d49f83-df5a-46d3-a33f-3bddbddc813b service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.777249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d49f83-df5a-46d3-a33f-3bddbddc813b service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.777512 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d49f83-df5a-46d3-a33f-3bddbddc813b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.779300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c6d49f83-df5a-46d3-a33f-3bddbddc813b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:35.790461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d49f83-df5a-46d3-a33f-3bddbddc813b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.791914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d49f83-df5a-46d3-a33f-3bddbddc813b service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.800769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1182/2350] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:35.988384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7098b8d-ea67-44da-ae60-cec11bb1a26b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.988973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7098b8d-ea67-44da-ae60-cec11bb1a26b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.988973 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7098b8d-ea67-44da-ae60-cec11bb1a26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:36.013039 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c7098b8d-ea67-44da-ae60-cec11bb1a26b None None] MFA Rules not processed for user `ae1cd97c879141e2b4c1527b5964d054`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:36.014360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c7098b8d-ea67-44da-ae60-cec11bb1a26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:36.073045 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1169/2351] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:36.136553 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d49f83-df5a-46d3-a33f-3bddbddc813b service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:36.139086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c6d49f83-df5a-46d3-a33f-3bddbddc813b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:36.161195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:36.221630 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d49f83-df5a-46d3-a33f-3bddbddc813b service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:36.226844 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfcaea-5b97-4e93-9fe1-20c7034e582c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:36.227456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfcaea-5b97-4e93-9fe1-20c7034e582c service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:36.227733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfcaea-5b97-4e93-9fe1-20c7034e582c service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:36.228026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfcaea-5b97-4e93-9fe1-20c7034e582c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:36.229870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-04dfcaea-5b97-4e93-9fe1-20c7034e582c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:36.242640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfcaea-5b97-4e93-9fe1-20c7034e582c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:36.244303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfcaea-5b97-4e93-9fe1-20c7034e582c service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:36.254702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1183/2352] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:36.439532 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7098b8d-ea67-44da-ae60-cec11bb1a26b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:36.450037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7098b8d-ea67-44da-ae60-cec11bb1a26b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:36.454664 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844af99-4593-4e55-aa4d-ac1f35e1898c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:36.455276 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844af99-4593-4e55-aa4d-ac1f35e1898c service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:36.455654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844af99-4593-4e55-aa4d-ac1f35e1898c service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:36.455807 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844af99-4593-4e55-aa4d-ac1f35e1898c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:36.457623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4844af99-4593-4e55-aa4d-ac1f35e1898c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:36.465211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:36.470989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844af99-4593-4e55-aa4d-ac1f35e1898c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:36.473625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844af99-4593-4e55-aa4d-ac1f35e1898c service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:36.484406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1170/2353] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:38:38.142189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfcaea-5b97-4e93-9fe1-20c7034e582c service placement] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:38.215507 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfcaea-5b97-4e93-9fe1-20c7034e582c service placement] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:38.222951 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:38.223644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:38.223925 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:38.224213 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] PATH_INFO: `/v3/users/e9385e35bc7c462c87cf9e4092708bd9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:38.225702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9385e35bc7c462c87cf9e4092708bd9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:38.227965 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:38.288593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9385e35bc7c462c87cf9e4092708bd9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:38.289292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9385e35bc7c462c87cf9e4092708bd9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:38.289906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9385e35bc7c462c87cf9e4092708bd9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:38.307930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9385e35bc7c462c87cf9e4092708bd9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:38.317560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9385e35bc7c462c87cf9e4092708bd9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:38.331491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1184/2354] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:38 2022] DELETE /identity/v3/users/e9385e35bc7c462c87cf9e4092708bd9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:38.336324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d78890-832e-4f56-8d75-ea0a47ca1437 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:38.336824 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d78890-832e-4f56-8d75-ea0a47ca1437 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:38.337117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d78890-832e-4f56-8d75-ea0a47ca1437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:38.367223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-70d78890-832e-4f56-8d75-ea0a47ca1437 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:38.368613 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-70d78890-832e-4f56-8d75-ea0a47ca1437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:38.438308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1171/2355] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:38:38.452288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:38.453860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:38.489758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:38.523569 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e21189-7498-4751-a81c-1af06e4f7ce3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:38.527261 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923556a-2b08-4c2c-890d-7aed8b8d3e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:38.527907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923556a-2b08-4c2c-890d-7aed8b8d3e94 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:38.528185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923556a-2b08-4c2c-890d-7aed8b8d3e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:38.528450 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923556a-2b08-4c2c-890d-7aed8b8d3e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:38.539364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923556a-2b08-4c2c-890d-7aed8b8d3e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:38.540870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923556a-2b08-4c2c-890d-7aed8b8d3e94 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:38.552600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1185/2356] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:38.858448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d78890-832e-4f56-8d75-ea0a47ca1437 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:38.860583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-70d78890-832e-4f56-8d75-ea0a47ca1437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:38.898219 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d78890-832e-4f56-8d75-ea0a47ca1437 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:38.903204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c007089-6b12-40bf-98fa-06ad95a8dcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:38.903864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c007089-6b12-40bf-98fa-06ad95a8dcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:38.904137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c007089-6b12-40bf-98fa-06ad95a8dcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:38.904403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c007089-6b12-40bf-98fa-06ad95a8dcb0 admin admin] PATH_INFO: `/v3/projects/5d67f75cc6904cc889ca0f678df98b9c` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:38.907100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c007089-6b12-40bf-98fa-06ad95a8dcb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d67f75cc6904cc889ca0f678df98b9c)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:38.909092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c007089-6b12-40bf-98fa-06ad95a8dcb0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:38.964598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9c007089-6b12-40bf-98fa-06ad95a8dcb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d67f75cc6904cc889ca0f678df98b9c', 'request_id': 'req-9c007089-6b12-40bf-98fa-06ad95a8dcb0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:38.973368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9c007089-6b12-40bf-98fa-06ad95a8dcb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d67f75cc6904cc889ca0f678df98b9c', 'request_id': 'req-9c007089-6b12-40bf-98fa-06ad95a8dcb0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:38.989967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9c007089-6b12-40bf-98fa-06ad95a8dcb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d67f75cc6904cc889ca0f678df98b9c', 'request_id': 'req-9c007089-6b12-40bf-98fa-06ad95a8dcb0'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:39.002994 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9c007089-6b12-40bf-98fa-06ad95a8dcb0 admin admin] The token cache is being invalidate because project 5d67f75cc6904cc889ca0f678df98b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:39.003288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9c007089-6b12-40bf-98fa-06ad95a8dcb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:39.003971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9c007089-6b12-40bf-98fa-06ad95a8dcb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:39.005810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1172/2357] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:38 2022] DELETE /identity/v3/projects/5d67f75cc6904cc889ca0f678df98b9c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:39.010548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f302287-5aca-481e-8f86-02d5380be442 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.010836 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f302287-5aca-481e-8f86-02d5380be442 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.011113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f302287-5aca-481e-8f86-02d5380be442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.031833 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7f302287-5aca-481e-8f86-02d5380be442 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:39.033012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7f302287-5aca-481e-8f86-02d5380be442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:39.098959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1186/2358] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:39.108789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757e4cb-3ff9-48ad-b0ee-4e1009a5daff None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.109506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757e4cb-3ff9-48ad-b0ee-4e1009a5daff None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.110409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757e4cb-3ff9-48ad-b0ee-4e1009a5daff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.135601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1757e4cb-3ff9-48ad-b0ee-4e1009a5daff None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:39.136914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1757e4cb-3ff9-48ad-b0ee-4e1009a5daff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:39.147308 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1173/2359] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:38:39.151806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f302287-5aca-481e-8f86-02d5380be442 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:39.163242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f302287-5aca-481e-8f86-02d5380be442 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:39.168548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de407cec-fda6-4648-9b0a-004a628de829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:39.169458 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de407cec-fda6-4648-9b0a-004a628de829 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.170138 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de407cec-fda6-4648-9b0a-004a628de829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.170389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de407cec-fda6-4648-9b0a-004a628de829 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.171266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de407cec-fda6-4648-9b0a-004a628de829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:38:39.171547 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de407cec-fda6-4648-9b0a-004a628de829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:39.173678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de407cec-fda6-4648-9b0a-004a628de829 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:39.186593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1187/2360] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:38:39.191622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757e4cb-3ff9-48ad-b0ee-4e1009a5daff None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:39.201877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757e4cb-3ff9-48ad-b0ee-4e1009a5daff None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:39.207046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13ddc2-e456-48dd-82b0-f5b285476708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:39.207652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13ddc2-e456-48dd-82b0-f5b285476708 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.207929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13ddc2-e456-48dd-82b0-f5b285476708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.208193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13ddc2-e456-48dd-82b0-f5b285476708 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.208934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13ddc2-e456-48dd-82b0-f5b285476708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:39.211067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13ddc2-e456-48dd-82b0-f5b285476708 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:39.246857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1174/2361] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:39 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:39.251565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de407cec-fda6-4648-9b0a-004a628de829 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:39.261474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de407cec-fda6-4648-9b0a-004a628de829 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:39.326165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b747a-e389-485c-bb93-746819a2e484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:39.327058 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b747a-e389-485c-bb93-746819a2e484 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.327522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b747a-e389-485c-bb93-746819a2e484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.327918 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b747a-e389-485c-bb93-746819a2e484 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.328693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b747a-e389-485c-bb93-746819a2e484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:39.330812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b747a-e389-485c-bb93-746819a2e484 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:39.354158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1188/2362] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:39 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:38:39.359435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13ddc2-e456-48dd-82b0-f5b285476708 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:39.370041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13ddc2-e456-48dd-82b0-f5b285476708 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:39.375440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729ac47-585b-4d0f-ab94-635c73228689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:39.376076 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729ac47-585b-4d0f-ab94-635c73228689 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.376355 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729ac47-585b-4d0f-ab94-635c73228689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.376636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729ac47-585b-4d0f-ab94-635c73228689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.377337 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729ac47-585b-4d0f-ab94-635c73228689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:39.378991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729ac47-585b-4d0f-ab94-635c73228689 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:39.389710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1175/2363] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:39.394210 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b747a-e389-485c-bb93-746819a2e484 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:39.405275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b747a-e389-485c-bb93-746819a2e484 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:39.410385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e28101-729b-418e-acac-2699e15b80a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:39.410991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e28101-729b-418e-acac-2699e15b80a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.411271 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e28101-729b-418e-acac-2699e15b80a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.411545 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e28101-729b-418e-acac-2699e15b80a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.412264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e28101-729b-418e-acac-2699e15b80a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:39.413851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e28101-729b-418e-acac-2699e15b80a9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:39.425721 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1189/2364] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:39.430043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729ac47-585b-4d0f-ab94-635c73228689 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:39.441074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729ac47-585b-4d0f-ab94-635c73228689 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:39.445854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd037f-6da1-4262-a817-67cf32c5bc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:39.446656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd037f-6da1-4262-a817-67cf32c5bc35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.446941 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd037f-6da1-4262-a817-67cf32c5bc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.447240 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd037f-6da1-4262-a817-67cf32c5bc35 admin admin] PATH_INFO: `/v3/projects/b59aae26f6f04c128bca151710e63c1d/users/383a83a108704d779997addcb1860b14/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.463141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd037f-6da1-4262-a817-67cf32c5bc35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b59aae26f6f04c128bca151710e63c1d, user_id=383a83a108704d779997addcb1860b14, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:39.466137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd037f-6da1-4262-a817-67cf32c5bc35 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:39.479940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1176/2365] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:39 2022] PUT /identity/v3/projects/b59aae26f6f04c128bca151710e63c1d/users/383a83a108704d779997addcb1860b14/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:39.490744 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af4e9f-3794-4ca0-b5ad-6c888217e015 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.491045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af4e9f-3794-4ca0-b5ad-6c888217e015 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.491300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af4e9f-3794-4ca0-b5ad-6c888217e015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.522652 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d8af4e9f-3794-4ca0-b5ad-6c888217e015 None None] MFA Rules not processed for user `383a83a108704d779997addcb1860b14`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:39.524037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d8af4e9f-3794-4ca0-b5ad-6c888217e015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:39.545911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:39.594753 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1190/2366] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:39.599027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd037f-6da1-4262-a817-67cf32c5bc35 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:39.614610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:39.669087 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd037f-6da1-4262-a817-67cf32c5bc35 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:39.674533 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ebf68-6df6-4932-b1a3-963fe09faf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:39.675137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ebf68-6df6-4932-b1a3-963fe09faf01 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.675412 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ebf68-6df6-4932-b1a3-963fe09faf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.675682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ebf68-6df6-4932-b1a3-963fe09faf01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.676306 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ebf68-6df6-4932-b1a3-963fe09faf01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:39.678269 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ebf68-6df6-4932-b1a3-963fe09faf01 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:39.711770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1177/2367] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:39 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:39.716044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af4e9f-3794-4ca0-b5ad-6c888217e015 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:39.728073 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af4e9f-3794-4ca0-b5ad-6c888217e015 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:39.800589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf81fb-a822-40d4-8714-3a68ec09bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:39.801158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf81fb-a822-40d4-8714-3a68ec09bd39 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.801440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf81fb-a822-40d4-8714-3a68ec09bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.801709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf81fb-a822-40d4-8714-3a68ec09bd39 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.802357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf81fb-a822-40d4-8714-3a68ec09bd39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:39.804323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf81fb-a822-40d4-8714-3a68ec09bd39 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:39.826541 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1191/2368] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:39 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:39.831139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ebf68-6df6-4932-b1a3-963fe09faf01 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:39.842114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ebf68-6df6-4932-b1a3-963fe09faf01 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:39.847508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27b006-cf39-496c-8c4d-e2203ce57f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:39.848116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27b006-cf39-496c-8c4d-e2203ce57f90 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.848392 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27b006-cf39-496c-8c4d-e2203ce57f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.848679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27b006-cf39-496c-8c4d-e2203ce57f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.849351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27b006-cf39-496c-8c4d-e2203ce57f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:39.850960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27b006-cf39-496c-8c4d-e2203ce57f90 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:39.863167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1178/2369] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:39.867555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf81fb-a822-40d4-8714-3a68ec09bd39 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:39.878142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf81fb-a822-40d4-8714-3a68ec09bd39 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:39.882842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d541d2-9b01-496c-99de-090e87443847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:39.883437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d541d2-9b01-496c-99de-090e87443847 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.883714 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d541d2-9b01-496c-99de-090e87443847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.883972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d541d2-9b01-496c-99de-090e87443847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.884659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d541d2-9b01-496c-99de-090e87443847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:39.886291 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d541d2-9b01-496c-99de-090e87443847 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:39.897110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1192/2370] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:39.901573 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27b006-cf39-496c-8c4d-e2203ce57f90 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:39.913195 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27b006-cf39-496c-8c4d-e2203ce57f90 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:39.919014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366d850-3e58-4528-a27f-667460cefc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:39.919674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366d850-3e58-4528-a27f-667460cefc6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.919948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366d850-3e58-4528-a27f-667460cefc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.920211 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366d850-3e58-4528-a27f-667460cefc6f admin admin] PATH_INFO: `/v3/projects/db3beb70dcd04b76aebb36faa1672ffa/users/f3b6dfaae07b416b8b420cb4b2755ee5/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:39.941247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366d850-3e58-4528-a27f-667460cefc6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3beb70dcd04b76aebb36faa1672ffa, user_id=f3b6dfaae07b416b8b420cb4b2755ee5, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:39.944283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366d850-3e58-4528-a27f-667460cefc6f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:39.968133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1179/2371] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:39 2022] PUT /identity/v3/projects/db3beb70dcd04b76aebb36faa1672ffa/users/f3b6dfaae07b416b8b420cb4b2755ee5/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:39.971938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76b3fa-3d9a-4fb0-8f7c-6e7622d3f181 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:39.972300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76b3fa-3d9a-4fb0-8f7c-6e7622d3f181 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:39.972672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76b3fa-3d9a-4fb0-8f7c-6e7622d3f181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:40.015793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-dd76b3fa-3d9a-4fb0-8f7c-6e7622d3f181 None None] MFA Rules not processed for user `f3b6dfaae07b416b8b420cb4b2755ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:40.017084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dd76b3fa-3d9a-4fb0-8f7c-6e7622d3f181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:40.095796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1193/2372] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:40.100779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366d850-3e58-4528-a27f-667460cefc6f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:40.191551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366d850-3e58-4528-a27f-667460cefc6f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:40.196446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed29e8-cabd-4da4-8e52-dbe59ffee9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:40.197042 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed29e8-cabd-4da4-8e52-dbe59ffee9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:40.197665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed29e8-cabd-4da4-8e52-dbe59ffee9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:40.197950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed29e8-cabd-4da4-8e52-dbe59ffee9a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:40.198698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed29e8-cabd-4da4-8e52-dbe59ffee9a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:40.200593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed29e8-cabd-4da4-8e52-dbe59ffee9a4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:40.236329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1180/2373] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:40 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:38:40.240231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76b3fa-3d9a-4fb0-8f7c-6e7622d3f181 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:40.257226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76b3fa-3d9a-4fb0-8f7c-6e7622d3f181 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:40.296449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:40.335152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f24c7-3424-48dd-95ea-2d0654197a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:40.335835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f24c7-3424-48dd-95ea-2d0654197a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:40.336235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f24c7-3424-48dd-95ea-2d0654197a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:40.336502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f24c7-3424-48dd-95ea-2d0654197a55 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:40.337273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f24c7-3424-48dd-95ea-2d0654197a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:40.339218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f24c7-3424-48dd-95ea-2d0654197a55 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:40.374221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1194/2374] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:40 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:40.375166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed29e8-cabd-4da4-8e52-dbe59ffee9a4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:40.387729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed29e8-cabd-4da4-8e52-dbe59ffee9a4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:40.396521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c5dfa-9577-4396-b05e-7ec9a4a20028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:40.397369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c5dfa-9577-4396-b05e-7ec9a4a20028 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:40.397900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c5dfa-9577-4396-b05e-7ec9a4a20028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:40.398228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c5dfa-9577-4396-b05e-7ec9a4a20028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:40.398955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c5dfa-9577-4396-b05e-7ec9a4a20028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:40.400780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c5dfa-9577-4396-b05e-7ec9a4a20028 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:40.412490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1181/2375] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:38:40.416900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f24c7-3424-48dd-95ea-2d0654197a55 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:40.428777 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f24c7-3424-48dd-95ea-2d0654197a55 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:40.433571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf49cc-0735-4243-bfe4-59ccb2b51d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:40.434267 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf49cc-0735-4243-bfe4-59ccb2b51d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:40.434484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf49cc-0735-4243-bfe4-59ccb2b51d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:40.434750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf49cc-0735-4243-bfe4-59ccb2b51d84 admin admin] PATH_INFO: `/v3/projects/b3ad5137572c4b66bf108aa49d4a7120/users/7020af66cca542cc9aa7e11ad9ee3ab9/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:40.453786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf49cc-0735-4243-bfe4-59ccb2b51d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ad5137572c4b66bf108aa49d4a7120, user_id=7020af66cca542cc9aa7e11ad9ee3ab9, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:40.471885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf49cc-0735-4243-bfe4-59ccb2b51d84 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:40.491991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1195/2376] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:40 2022] PUT /identity/v3/projects/b3ad5137572c4b66bf108aa49d4a7120/users/7020af66cca542cc9aa7e11ad9ee3ab9/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:40.493069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c5dfa-9577-4396-b05e-7ec9a4a20028 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:40.505252 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c5dfa-9577-4396-b05e-7ec9a4a20028 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:40.549260 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:40.596630 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5498a0-b87a-4cc5-88d3-fcfc68fadba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:40.597369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5498a0-b87a-4cc5-88d3-fcfc68fadba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:40.597658 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5498a0-b87a-4cc5-88d3-fcfc68fadba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:40.598031 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5498a0-b87a-4cc5-88d3-fcfc68fadba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:40.598663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5498a0-b87a-4cc5-88d3-fcfc68fadba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:40.600342 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5498a0-b87a-4cc5-88d3-fcfc68fadba7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:40.614271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1182/2377] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:40.618127 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf49cc-0735-4243-bfe4-59ccb2b51d84 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:40.633280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf49cc-0735-4243-bfe4-59ccb2b51d84 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:40.641606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb1f7a-26bf-4f53-b1be-06907cc3632c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:40.642334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb1f7a-26bf-4f53-b1be-06907cc3632c admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:40.642604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb1f7a-26bf-4f53-b1be-06907cc3632c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:40.643084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb1f7a-26bf-4f53-b1be-06907cc3632c admin admin] PATH_INFO: `/v3/projects/b3ad5137572c4b66bf108aa49d4a7120/users/7020af66cca542cc9aa7e11ad9ee3ab9/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:40.645401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb1f7a-26bf-4f53-b1be-06907cc3632c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ad5137572c4b66bf108aa49d4a7120, user_id=7020af66cca542cc9aa7e11ad9ee3ab9, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:40.649641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb1f7a-26bf-4f53-b1be-06907cc3632c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:40.668057 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1196/2378] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:40 2022] PUT /identity/v3/projects/b3ad5137572c4b66bf108aa49d4a7120/users/7020af66cca542cc9aa7e11ad9ee3ab9/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:38:40.680053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7e878-a32a-4668-8d25-8f92b824359b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:40.680377 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7e878-a32a-4668-8d25-8f92b824359b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:40.680609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7e878-a32a-4668-8d25-8f92b824359b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:40.713390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-97f7e878-a32a-4668-8d25-8f92b824359b None None] MFA Rules not processed for user `7020af66cca542cc9aa7e11ad9ee3ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:40.714592 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-97f7e878-a32a-4668-8d25-8f92b824359b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:40.794288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1183/2379] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:40.806767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb1f7a-26bf-4f53-b1be-06907cc3632c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:40.808798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-79eb1f7a-26bf-4f53-b1be-06907cc3632c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:40.870688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb1f7a-26bf-4f53-b1be-06907cc3632c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:40.874906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e03fed-8b87-4997-931a-2d24b5d3977e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:40.875525 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e03fed-8b87-4997-931a-2d24b5d3977e service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:40.875805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e03fed-8b87-4997-931a-2d24b5d3977e service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:40.876075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e03fed-8b87-4997-931a-2d24b5d3977e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:40.885969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e03fed-8b87-4997-931a-2d24b5d3977e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:40.887404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e03fed-8b87-4997-931a-2d24b5d3977e service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:40.898061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1197/2380] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:41.477704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7e878-a32a-4668-8d25-8f92b824359b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:41.479281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-97f7e878-a32a-4668-8d25-8f92b824359b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:41.491991 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7e878-a32a-4668-8d25-8f92b824359b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:41.495761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db297d21-6cee-489f-a410-8aca763b3e99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:41.496356 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db297d21-6cee-489f-a410-8aca763b3e99 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:41.496633 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db297d21-6cee-489f-a410-8aca763b3e99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:41.496942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db297d21-6cee-489f-a410-8aca763b3e99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:41.557217 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db297d21-6cee-489f-a410-8aca763b3e99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:41.558650 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db297d21-6cee-489f-a410-8aca763b3e99 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:41.568127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1184/2381] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:45.686670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e03fed-8b87-4997-931a-2d24b5d3977e service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:45.695801 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e03fed-8b87-4997-931a-2d24b5d3977e service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:45.699701 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55139395-d1e7-40f4-902c-4eed0e2b4a32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:45.700283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55139395-d1e7-40f4-902c-4eed0e2b4a32 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:45.700544 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55139395-d1e7-40f4-902c-4eed0e2b4a32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:45.700842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55139395-d1e7-40f4-902c-4eed0e2b4a32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:45.765782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55139395-d1e7-40f4-902c-4eed0e2b4a32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:45.767124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55139395-d1e7-40f4-902c-4eed0e2b4a32 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:45.777355 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1198/2382] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:46.734150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db297d21-6cee-489f-a410-8aca763b3e99 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:46.735868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-db297d21-6cee-489f-a410-8aca763b3e99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:46.821177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:46.840526 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db297d21-6cee-489f-a410-8aca763b3e99 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:46.846251 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786ee7d-6f91-40e3-83db-411711447d64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:46.846859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786ee7d-6f91-40e3-83db-411711447d64 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:46.847122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786ee7d-6f91-40e3-83db-411711447d64 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:46.847388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786ee7d-6f91-40e3-83db-411711447d64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:46.849278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7786ee7d-6f91-40e3-83db-411711447d64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:46.864169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786ee7d-6f91-40e3-83db-411711447d64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:46.865840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786ee7d-6f91-40e3-83db-411711447d64 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:46.877127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1185/2383] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:47.981200 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55139395-d1e7-40f4-902c-4eed0e2b4a32 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:48.008761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:48.045999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55139395-d1e7-40f4-902c-4eed0e2b4a32 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:48.051671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:48.052284 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:48.052559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:48.052850 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] PATH_INFO: `/v3/users/383a83a108704d779997addcb1860b14` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:48.054041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=383a83a108704d779997addcb1860b14)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:48.055939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:48.114175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '383a83a108704d779997addcb1860b14'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.114757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '383a83a108704d779997addcb1860b14'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.115400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '383a83a108704d779997addcb1860b14'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.127947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '383a83a108704d779997addcb1860b14'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.137083 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '383a83a108704d779997addcb1860b14'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.150490 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1199/2384] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:47 2022] DELETE /identity/v3/users/383a83a108704d779997addcb1860b14 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:48.155529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bff3d-bcab-490d-9066-67d62eea123c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:48.156041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bff3d-bcab-490d-9066-67d62eea123c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:48.156041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bff3d-bcab-490d-9066-67d62eea123c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:48.176163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-773bff3d-bcab-490d-9066-67d62eea123c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:48.177400 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-773bff3d-bcab-490d-9066-67d62eea123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:48.239517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1186/2385] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:48.252218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:48.253751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:48.318093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82639d19-dede-4f0a-b21e-4610ef0a1210 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:48.323224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897ddb5-39e0-40a9-b36e-2d59ff0ef8e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:48.323837 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897ddb5-39e0-40a9-b36e-2d59ff0ef8e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:48.324142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897ddb5-39e0-40a9-b36e-2d59ff0ef8e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:48.324512 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897ddb5-39e0-40a9-b36e-2d59ff0ef8e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:48.335242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897ddb5-39e0-40a9-b36e-2d59ff0ef8e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:48.336677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897ddb5-39e0-40a9-b36e-2d59ff0ef8e3 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:48.347952 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1200/2386] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:38:48.630234 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bff3d-bcab-490d-9066-67d62eea123c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:48.631792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-773bff3d-bcab-490d-9066-67d62eea123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:48.645269 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bff3d-bcab-490d-9066-67d62eea123c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:48.650819 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:48.651662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:48.652022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:48.652445 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] PATH_INFO: `/v3/projects/b59aae26f6f04c128bca151710e63c1d` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:48.653655 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b59aae26f6f04c128bca151710e63c1d)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:48.655878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:48.707516 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b59aae26f6f04c128bca151710e63c1d', 'request_id': 'req-4d89884d-aeb4-4c31-ae71-d95d10935fa9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.716746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b59aae26f6f04c128bca151710e63c1d', 'request_id': 'req-4d89884d-aeb4-4c31-ae71-d95d10935fa9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.727372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b59aae26f6f04c128bca151710e63c1d', 'request_id': 'req-4d89884d-aeb4-4c31-ae71-d95d10935fa9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.741005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] The token cache is being invalidate because project b59aae26f6f04c128bca151710e63c1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:48.741292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.742023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.743611 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1187/2387] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:48 2022] DELETE /identity/v3/projects/b59aae26f6f04c128bca151710e63c1d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:48.747846 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897ddb5-39e0-40a9-b36e-2d59ff0ef8e3 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:48.757934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897ddb5-39e0-40a9-b36e-2d59ff0ef8e3 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:48.803432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:48.818218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:48.818876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:48.819152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:48.819422 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] PATH_INFO: `/v3/users/f3b6dfaae07b416b8b420cb4b2755ee5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:48.820548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b6dfaae07b416b8b420cb4b2755ee5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:48.823176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:48.880098 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b6dfaae07b416b8b420cb4b2755ee5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.881358 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b6dfaae07b416b8b420cb4b2755ee5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.882266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b6dfaae07b416b8b420cb4b2755ee5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.898383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b6dfaae07b416b8b420cb4b2755ee5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.910234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b6dfaae07b416b8b420cb4b2755ee5'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:48.929973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1201/2388] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:48 2022] DELETE /identity/v3/users/f3b6dfaae07b416b8b420cb4b2755ee5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:49.360066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:49.361647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:49.385730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89884d-aeb4-4c31-ae71-d95d10935fa9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:49.454098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:49.454739 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:49.455018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:49.455292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] PATH_INFO: `/v3/projects/db3beb70dcd04b76aebb36faa1672ffa` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:49.456439 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db3beb70dcd04b76aebb36faa1672ffa)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:49.458410 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:49.468136 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:49.513241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3beb70dcd04b76aebb36faa1672ffa', 'request_id': 'req-732c503c-f06e-4504-ae13-9e2ddaedec62'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:49.521314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3beb70dcd04b76aebb36faa1672ffa', 'request_id': 'req-732c503c-f06e-4504-ae13-9e2ddaedec62'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:49.527798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3beb70dcd04b76aebb36faa1672ffa', 'request_id': 'req-732c503c-f06e-4504-ae13-9e2ddaedec62'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:49.538693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] The token cache is being invalidate because project db3beb70dcd04b76aebb36faa1672ffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:49.538981 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:49.539667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:49.541341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1188/2389] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:49 2022] DELETE /identity/v3/projects/db3beb70dcd04b76aebb36faa1672ffa => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:49.545564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:49.547927 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:49.561661 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c5dd3-f8ed-4c9b-b6f3-4f213f364469 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:49.623074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae628a3-3ee1-464a-ae29-6369920d748e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:49.623796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae628a3-3ee1-464a-ae29-6369920d748e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:49.624148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae628a3-3ee1-464a-ae29-6369920d748e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:49.624489 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae628a3-3ee1-464a-ae29-6369920d748e admin admin] PATH_INFO: `/v3/users/7020af66cca542cc9aa7e11ad9ee3ab9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:49.625697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae628a3-3ee1-464a-ae29-6369920d748e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7020af66cca542cc9aa7e11ad9ee3ab9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:49.627812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae628a3-3ee1-464a-ae29-6369920d748e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:49.678788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7ae628a3-3ee1-464a-ae29-6369920d748e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7020af66cca542cc9aa7e11ad9ee3ab9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:49.679486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7ae628a3-3ee1-464a-ae29-6369920d748e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7020af66cca542cc9aa7e11ad9ee3ab9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:49.680089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7ae628a3-3ee1-464a-ae29-6369920d748e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7020af66cca542cc9aa7e11ad9ee3ab9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:49.694176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7ae628a3-3ee1-464a-ae29-6369920d748e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7020af66cca542cc9aa7e11ad9ee3ab9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:49.696980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:49.704466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7ae628a3-3ee1-464a-ae29-6369920d748e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7020af66cca542cc9aa7e11ad9ee3ab9'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:49.718897 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1202/2390] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:49 2022] DELETE /identity/v3/users/7020af66cca542cc9aa7e11ad9ee3ab9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:50.351964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:50.353458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:50.366590 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c503c-f06e-4504-ae13-9e2ddaedec62 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:50.429235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:50.432362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:50.432362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:50.432362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b admin admin] PATH_INFO: `/v3/projects/b3ad5137572c4b66bf108aa49d4a7120` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:50.432362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ad5137572c4b66bf108aa49d4a7120)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:50.433456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:50.505021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ad5137572c4b66bf108aa49d4a7120', 'request_id': 'req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:50.517002 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ad5137572c4b66bf108aa49d4a7120', 'request_id': 'req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:50.525653 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ad5137572c4b66bf108aa49d4a7120', 'request_id': 'req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:50.537497 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b admin admin] The token cache is being invalidate because project b3ad5137572c4b66bf108aa49d4a7120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:50.537861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:50.538430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bdd0ed1a-b07e-4b56-b1d1-a346106cc52b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:50.540171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1189/2391] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:50 2022] DELETE /identity/v3/projects/b3ad5137572c4b66bf108aa49d4a7120 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:50.546886 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43ac4a-e995-41ed-b81c-c2b4fb63e184 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:50.547438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43ac4a-e995-41ed-b81c-c2b4fb63e184 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:50.547959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43ac4a-e995-41ed-b81c-c2b4fb63e184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:50.585998 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2d43ac4a-e995-41ed-b81c-c2b4fb63e184 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:50.587239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2d43ac4a-e995-41ed-b81c-c2b4fb63e184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:50.655494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1203/2392] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:50.664274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036df15-de2f-411e-b4cc-0948372143ae None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:50.664477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036df15-de2f-411e-b4cc-0948372143ae None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:50.664712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036df15-de2f-411e-b4cc-0948372143ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:50.686943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3036df15-de2f-411e-b4cc-0948372143ae None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:50.688093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3036df15-de2f-411e-b4cc-0948372143ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:50.696959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1190/2393] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:50.701540 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43ac4a-e995-41ed-b81c-c2b4fb63e184 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:50.711990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43ac4a-e995-41ed-b81c-c2b4fb63e184 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:50.717536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b23f93-a32d-404b-a8fd-e3b85aae31ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:50.718149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b23f93-a32d-404b-a8fd-e3b85aae31ca admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:50.718433 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b23f93-a32d-404b-a8fd-e3b85aae31ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:50.718688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b23f93-a32d-404b-a8fd-e3b85aae31ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:50.719349 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b23f93-a32d-404b-a8fd-e3b85aae31ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:38:50.719666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b23f93-a32d-404b-a8fd-e3b85aae31ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:50.721317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b23f93-a32d-404b-a8fd-e3b85aae31ca admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:50.740707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1204/2394] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:38:50.744926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036df15-de2f-411e-b4cc-0948372143ae None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:50.755111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036df15-de2f-411e-b4cc-0948372143ae None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:50.759848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f7a53-730f-4159-af57-9bcfa9953d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:50.760511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f7a53-730f-4159-af57-9bcfa9953d40 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:50.760881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f7a53-730f-4159-af57-9bcfa9953d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:50.761292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f7a53-730f-4159-af57-9bcfa9953d40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:50.761987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f7a53-730f-4159-af57-9bcfa9953d40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:50.764157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f7a53-730f-4159-af57-9bcfa9953d40 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:50.789679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:50.800108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1191/2395] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:50 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:38:50.803964 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b23f93-a32d-404b-a8fd-e3b85aae31ca admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:50.814704 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b23f93-a32d-404b-a8fd-e3b85aae31ca admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:50.880831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df09b8-c744-4e60-8cd6-31cebdd5cbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:50.881439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df09b8-c744-4e60-8cd6-31cebdd5cbf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:50.881713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df09b8-c744-4e60-8cd6-31cebdd5cbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:50.882035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df09b8-c744-4e60-8cd6-31cebdd5cbf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:50.882664 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df09b8-c744-4e60-8cd6-31cebdd5cbf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:50.884685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df09b8-c744-4e60-8cd6-31cebdd5cbf3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:50.908064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1205/2396] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:50 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:38:50.912395 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f7a53-730f-4159-af57-9bcfa9953d40 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:50.924441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f7a53-730f-4159-af57-9bcfa9953d40 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:50.933105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71b8ab-576f-486d-8be4-49b5a6188fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:50.933742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71b8ab-576f-486d-8be4-49b5a6188fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:50.934018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71b8ab-576f-486d-8be4-49b5a6188fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:50.934286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71b8ab-576f-486d-8be4-49b5a6188fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:50.934943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71b8ab-576f-486d-8be4-49b5a6188fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:50.937388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71b8ab-576f-486d-8be4-49b5a6188fc3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:50.950768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1192/2397] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:50.955578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df09b8-c744-4e60-8cd6-31cebdd5cbf3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:50.966118 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df09b8-c744-4e60-8cd6-31cebdd5cbf3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:50.971270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015a683-ea85-41e5-8553-2b2dc3435728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:50.971932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015a683-ea85-41e5-8553-2b2dc3435728 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:50.972336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015a683-ea85-41e5-8553-2b2dc3435728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:50.973195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015a683-ea85-41e5-8553-2b2dc3435728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:50.978406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015a683-ea85-41e5-8553-2b2dc3435728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:50.980310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015a683-ea85-41e5-8553-2b2dc3435728 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:51.002207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1206/2398] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:51.003040 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71b8ab-576f-486d-8be4-49b5a6188fc3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:51.014881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71b8ab-576f-486d-8be4-49b5a6188fc3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:51.020572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0e07e-d7dd-4d82-98d1-7659bdd0ed36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:51.021273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0e07e-d7dd-4d82-98d1-7659bdd0ed36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:51.021553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0e07e-d7dd-4d82-98d1-7659bdd0ed36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:51.021825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0e07e-d7dd-4d82-98d1-7659bdd0ed36 admin admin] PATH_INFO: `/v3/projects/7454ff149b7e446a823b7a1089cd6407/users/397ab128ff2b4cd593982cf64953c409/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:51.039916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0e07e-d7dd-4d82-98d1-7659bdd0ed36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7454ff149b7e446a823b7a1089cd6407, user_id=397ab128ff2b4cd593982cf64953c409, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:51.042630 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0e07e-d7dd-4d82-98d1-7659bdd0ed36 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:51.070802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1193/2399] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:51 2022] PUT /identity/v3/projects/7454ff149b7e446a823b7a1089cd6407/users/397ab128ff2b4cd593982cf64953c409/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:51.083503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876a092-fe29-423c-ac53-54d14cf397dc None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:51.083805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876a092-fe29-423c-ac53-54d14cf397dc None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:51.084132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876a092-fe29-423c-ac53-54d14cf397dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:51.116445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c876a092-fe29-423c-ac53-54d14cf397dc None None] MFA Rules not processed for user `397ab128ff2b4cd593982cf64953c409`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:51.118060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c876a092-fe29-423c-ac53-54d14cf397dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:51.168939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:51.203885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1207/2400] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:51.208727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0e07e-d7dd-4d82-98d1-7659bdd0ed36 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:51.274038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0e07e-d7dd-4d82-98d1-7659bdd0ed36 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:51.278892 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a3951-2291-4f7f-813d-2f45aab98c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:51.279768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a3951-2291-4f7f-813d-2f45aab98c8d admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:51.279947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a3951-2291-4f7f-813d-2f45aab98c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:51.280132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a3951-2291-4f7f-813d-2f45aab98c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:51.280806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a3951-2291-4f7f-813d-2f45aab98c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:51.282702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a3951-2291-4f7f-813d-2f45aab98c8d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:51.314785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1194/2401] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:51 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:51.316275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876a092-fe29-423c-ac53-54d14cf397dc None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:51.327558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876a092-fe29-423c-ac53-54d14cf397dc None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:51.395724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e03757-c323-4bc7-a3b9-714bc98e66e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:51.396430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e03757-c323-4bc7-a3b9-714bc98e66e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:51.396822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e03757-c323-4bc7-a3b9-714bc98e66e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:51.397993 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e03757-c323-4bc7-a3b9-714bc98e66e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:51.398819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e03757-c323-4bc7-a3b9-714bc98e66e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:51.401326 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e03757-c323-4bc7-a3b9-714bc98e66e5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:51.438251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1208/2402] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:51 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:51.442241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a3951-2291-4f7f-813d-2f45aab98c8d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:51.457105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a3951-2291-4f7f-813d-2f45aab98c8d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:51.462107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596337d6-b33f-4c0d-b131-da190a315a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:51.462793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596337d6-b33f-4c0d-b131-da190a315a11 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:51.463132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596337d6-b33f-4c0d-b131-da190a315a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:51.463472 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596337d6-b33f-4c0d-b131-da190a315a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:51.464317 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596337d6-b33f-4c0d-b131-da190a315a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:51.466936 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596337d6-b33f-4c0d-b131-da190a315a11 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:51.483990 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e03757-c323-4bc7-a3b9-714bc98e66e5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:51.484431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1195/2403] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:51.494396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e03757-c323-4bc7-a3b9-714bc98e66e5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:51.500274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad525c2-2f83-4cd4-ba2b-fbc6a97356b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:51.500992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad525c2-2f83-4cd4-ba2b-fbc6a97356b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:51.501267 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad525c2-2f83-4cd4-ba2b-fbc6a97356b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:51.501534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad525c2-2f83-4cd4-ba2b-fbc6a97356b5 admin admin] PATH_INFO: `/v3/projects/e27a3e2c2ca84d3fb22f5a2be4b292b1/users/2059932426734ebbb5e930a1d99aab10/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:51.519623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad525c2-2f83-4cd4-ba2b-fbc6a97356b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27a3e2c2ca84d3fb22f5a2be4b292b1, user_id=2059932426734ebbb5e930a1d99aab10, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:51.522332 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad525c2-2f83-4cd4-ba2b-fbc6a97356b5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:51.541915 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1209/2404] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:51 2022] PUT /identity/v3/projects/e27a3e2c2ca84d3fb22f5a2be4b292b1/users/2059932426734ebbb5e930a1d99aab10/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:51.546094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596337d6-b33f-4c0d-b131-da190a315a11 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:51.562023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596337d6-b33f-4c0d-b131-da190a315a11 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:51.630286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea5956-e11a-4369-b316-69160d6c1d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:51.630890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea5956-e11a-4369-b316-69160d6c1d80 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:51.631165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea5956-e11a-4369-b316-69160d6c1d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:51.631434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea5956-e11a-4369-b316-69160d6c1d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:51.632135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea5956-e11a-4369-b316-69160d6c1d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:51.633809 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea5956-e11a-4369-b316-69160d6c1d80 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:51.644113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1196/2405] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:51.648267 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad525c2-2f83-4cd4-ba2b-fbc6a97356b5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:51.659967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad525c2-2f83-4cd4-ba2b-fbc6a97356b5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:51.664995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3a961-c532-4464-a5e0-2fa4bd98e161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:51.665891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3a961-c532-4464-a5e0-2fa4bd98e161 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:51.666303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3a961-c532-4464-a5e0-2fa4bd98e161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:51.666401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3a961-c532-4464-a5e0-2fa4bd98e161 admin admin] PATH_INFO: `/v3/projects/e27a3e2c2ca84d3fb22f5a2be4b292b1/users/2059932426734ebbb5e930a1d99aab10/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:51.668330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3a961-c532-4464-a5e0-2fa4bd98e161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27a3e2c2ca84d3fb22f5a2be4b292b1, user_id=2059932426734ebbb5e930a1d99aab10, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:51.671378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3a961-c532-4464-a5e0-2fa4bd98e161 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:51.684189 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1210/2406] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:51 2022] PUT /identity/v3/projects/e27a3e2c2ca84d3fb22f5a2be4b292b1/users/2059932426734ebbb5e930a1d99aab10/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:51.695351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413d785-c521-4d5e-baf4-04fc21533a2a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:51.695636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413d785-c521-4d5e-baf4-04fc21533a2a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:51.696424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413d785-c521-4d5e-baf4-04fc21533a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:51.727296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b413d785-c521-4d5e-baf4-04fc21533a2a None None] MFA Rules not processed for user `2059932426734ebbb5e930a1d99aab10`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:51.728574 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b413d785-c521-4d5e-baf4-04fc21533a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:51.758008 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:51.806545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1197/2407] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:51.818258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3a961-c532-4464-a5e0-2fa4bd98e161 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:51.819926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-55a3a961-c532-4464-a5e0-2fa4bd98e161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:51.884203 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3a961-c532-4464-a5e0-2fa4bd98e161 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:51.888850 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1cf144-6e5c-4efb-ac84-a65c66643381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:51.889673 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cf144-6e5c-4efb-ac84-a65c66643381 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:51.889955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cf144-6e5c-4efb-ac84-a65c66643381 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:51.890227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cf144-6e5c-4efb-ac84-a65c66643381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:51.900289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1cf144-6e5c-4efb-ac84-a65c66643381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:51.901774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1cf144-6e5c-4efb-ac84-a65c66643381 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:51.912334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1211/2408] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:52.304348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413d785-c521-4d5e-baf4-04fc21533a2a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:52.381194 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413d785-c521-4d5e-baf4-04fc21533a2a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:52.386326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:52.386951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:52.387229 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:52.387502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] PATH_INFO: `/v3/users/397ab128ff2b4cd593982cf64953c409` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:52.388806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=397ab128ff2b4cd593982cf64953c409)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:52.390859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:52.453134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '397ab128ff2b4cd593982cf64953c409'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:52.453839 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '397ab128ff2b4cd593982cf64953c409'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:52.454559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '397ab128ff2b4cd593982cf64953c409'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:52.469046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '397ab128ff2b4cd593982cf64953c409'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:52.477850 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '397ab128ff2b4cd593982cf64953c409'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:52.496759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1198/2409] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:52 2022] DELETE /identity/v3/users/397ab128ff2b4cd593982cf64953c409 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:52.501938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7fb4b-2e97-4bde-b25c-ab7b0b709afd None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:52.502221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7fb4b-2e97-4bde-b25c-ab7b0b709afd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:52.502490 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7fb4b-2e97-4bde-b25c-ab7b0b709afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:52.523252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f5a7fb4b-2e97-4bde-b25c-ab7b0b709afd None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:52.524465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f5a7fb4b-2e97-4bde-b25c-ab7b0b709afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:52.555660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:52.596481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1212/2410] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:52.607348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:52.610124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:52.664110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:52.692676 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299971fa-0304-4ba6-b72f-d6822b8bb785 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:52.697385 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945d058-8811-4a13-bce7-1a55b504f070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:52.697983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945d058-8811-4a13-bce7-1a55b504f070 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:52.698261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945d058-8811-4a13-bce7-1a55b504f070 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:52.698534 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945d058-8811-4a13-bce7-1a55b504f070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:52.711300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945d058-8811-4a13-bce7-1a55b504f070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:52.712693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945d058-8811-4a13-bce7-1a55b504f070 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:52.724135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1199/2411] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:53.071522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7fb4b-2e97-4bde-b25c-ab7b0b709afd None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:53.073030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f5a7fb4b-2e97-4bde-b25c-ab7b0b709afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:53.086332 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7fb4b-2e97-4bde-b25c-ab7b0b709afd None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:53.091030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:53.091726 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:53.092068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:53.092302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] PATH_INFO: `/v3/projects/7454ff149b7e446a823b7a1089cd6407` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:53.093633 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7454ff149b7e446a823b7a1089cd6407)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:53.095536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:53.145674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7454ff149b7e446a823b7a1089cd6407', 'request_id': 'req-5fb8f558-51cd-4350-a53e-b338e23d0b43'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.154373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7454ff149b7e446a823b7a1089cd6407', 'request_id': 'req-5fb8f558-51cd-4350-a53e-b338e23d0b43'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.161147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7454ff149b7e446a823b7a1089cd6407', 'request_id': 'req-5fb8f558-51cd-4350-a53e-b338e23d0b43'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.173042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] The token cache is being invalidate because project 7454ff149b7e446a823b7a1089cd6407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:53.173365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.174137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.176027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1213/2412] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:53 2022] DELETE /identity/v3/projects/7454ff149b7e446a823b7a1089cd6407 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:53.179927 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945d058-8811-4a13-bce7-1a55b504f070 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:53.189869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945d058-8811-4a13-bce7-1a55b504f070 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:53.252290 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685071e-3e7c-40a0-8ed5-e83af41e2c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:53.253290 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685071e-3e7c-40a0-8ed5-e83af41e2c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:53.253639 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685071e-3e7c-40a0-8ed5-e83af41e2c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:53.253976 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685071e-3e7c-40a0-8ed5-e83af41e2c6f admin admin] PATH_INFO: `/v3/users/2059932426734ebbb5e930a1d99aab10` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:53.255193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685071e-3e7c-40a0-8ed5-e83af41e2c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2059932426734ebbb5e930a1d99aab10)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:53.257170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685071e-3e7c-40a0-8ed5-e83af41e2c6f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:53.307519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b685071e-3e7c-40a0-8ed5-e83af41e2c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2059932426734ebbb5e930a1d99aab10'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.308176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b685071e-3e7c-40a0-8ed5-e83af41e2c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2059932426734ebbb5e930a1d99aab10'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.308811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b685071e-3e7c-40a0-8ed5-e83af41e2c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2059932426734ebbb5e930a1d99aab10'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.322182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b685071e-3e7c-40a0-8ed5-e83af41e2c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2059932426734ebbb5e930a1d99aab10'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.331782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b685071e-3e7c-40a0-8ed5-e83af41e2c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2059932426734ebbb5e930a1d99aab10'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.346522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1200/2413] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:53 2022] DELETE /identity/v3/users/2059932426734ebbb5e930a1d99aab10 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:53.641683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:53.643368 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:53.658234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8f558-51cd-4350-a53e-b338e23d0b43 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:53.725887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b5b61-31ac-4410-b9e6-3802eaead823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:53.726519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b5b61-31ac-4410-b9e6-3802eaead823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:53.726800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b5b61-31ac-4410-b9e6-3802eaead823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:53.727071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b5b61-31ac-4410-b9e6-3802eaead823 admin admin] PATH_INFO: `/v3/projects/e27a3e2c2ca84d3fb22f5a2be4b292b1` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:53.728183 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b5b61-31ac-4410-b9e6-3802eaead823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e27a3e2c2ca84d3fb22f5a2be4b292b1)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:53.730140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b5b61-31ac-4410-b9e6-3802eaead823 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:53.782322 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-745b5b61-31ac-4410-b9e6-3802eaead823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e27a3e2c2ca84d3fb22f5a2be4b292b1', 'request_id': 'req-745b5b61-31ac-4410-b9e6-3802eaead823'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.790279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-745b5b61-31ac-4410-b9e6-3802eaead823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e27a3e2c2ca84d3fb22f5a2be4b292b1', 'request_id': 'req-745b5b61-31ac-4410-b9e6-3802eaead823'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.797911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-745b5b61-31ac-4410-b9e6-3802eaead823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e27a3e2c2ca84d3fb22f5a2be4b292b1', 'request_id': 'req-745b5b61-31ac-4410-b9e6-3802eaead823'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.799178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:53.811857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-745b5b61-31ac-4410-b9e6-3802eaead823 admin admin] The token cache is being invalidate because project e27a3e2c2ca84d3fb22f5a2be4b292b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:53.812429 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-745b5b61-31ac-4410-b9e6-3802eaead823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.813343 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-745b5b61-31ac-4410-b9e6-3802eaead823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:53.820237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1214/2414] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:53 2022] DELETE /identity/v3/projects/e27a3e2c2ca84d3fb22f5a2be4b292b1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:53.825287 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb2e8b-03d6-44f2-9ccb-38aa5b2427bf None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:53.825806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb2e8b-03d6-44f2-9ccb-38aa5b2427bf None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:53.826374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb2e8b-03d6-44f2-9ccb-38aa5b2427bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:53.847495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1ffb2e8b-03d6-44f2-9ccb-38aa5b2427bf None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:53.848748 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1ffb2e8b-03d6-44f2-9ccb-38aa5b2427bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:53.905563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:53.916283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1201/2415] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:53.925724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3beac9c-5cdf-4e2b-9845-2cc079ad5003 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:53.926012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3beac9c-5cdf-4e2b-9845-2cc079ad5003 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:53.926286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3beac9c-5cdf-4e2b-9845-2cc079ad5003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:53.946509 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a3beac9c-5cdf-4e2b-9845-2cc079ad5003 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:53.947749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a3beac9c-5cdf-4e2b-9845-2cc079ad5003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:53.956522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1215/2416] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:53.960962 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb2e8b-03d6-44f2-9ccb-38aa5b2427bf None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:53.971424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb2e8b-03d6-44f2-9ccb-38aa5b2427bf None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:53.976425 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f1f13-941d-4cb0-a144-629362c8d513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:53.977032 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f1f13-941d-4cb0-a144-629362c8d513 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:53.977305 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f1f13-941d-4cb0-a144-629362c8d513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:53.977573 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f1f13-941d-4cb0-a144-629362c8d513 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:53.978255 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f1f13-941d-4cb0-a144-629362c8d513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:38:53.978552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f1f13-941d-4cb0-a144-629362c8d513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:53.980265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f1f13-941d-4cb0-a144-629362c8d513 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:53.993365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1202/2417] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:38:53.998121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3beac9c-5cdf-4e2b-9845-2cc079ad5003 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:54.008091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3beac9c-5cdf-4e2b-9845-2cc079ad5003 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:54.014111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc4fd9-42cd-44df-880c-af50dc6d68d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:54.014718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc4fd9-42cd-44df-880c-af50dc6d68d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:54.014995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc4fd9-42cd-44df-880c-af50dc6d68d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:54.015260 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc4fd9-42cd-44df-880c-af50dc6d68d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:54.015881 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc4fd9-42cd-44df-880c-af50dc6d68d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:54.017837 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc4fd9-42cd-44df-880c-af50dc6d68d0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:54.047867 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1216/2418] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:53 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:54.052363 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f1f13-941d-4cb0-a144-629362c8d513 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:54.063104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f1f13-941d-4cb0-a144-629362c8d513 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:54.131841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659e3ce-aebc-46c7-a9bd-902e23545ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:54.132446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659e3ce-aebc-46c7-a9bd-902e23545ccb admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:54.132755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659e3ce-aebc-46c7-a9bd-902e23545ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:54.133028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659e3ce-aebc-46c7-a9bd-902e23545ccb admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:54.133645 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e659e3ce-aebc-46c7-a9bd-902e23545ccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:54.135575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e659e3ce-aebc-46c7-a9bd-902e23545ccb admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:54.157843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1203/2419] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:54 2022] POST /identity/v3/users => generated 281 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:38:54.161925 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc4fd9-42cd-44df-880c-af50dc6d68d0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:54.173323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc4fd9-42cd-44df-880c-af50dc6d68d0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:54.182114 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c7c24-e2d8-49f2-add1-80ab3c14a507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:54.182712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c7c24-e2d8-49f2-add1-80ab3c14a507 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:54.182983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c7c24-e2d8-49f2-add1-80ab3c14a507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:54.183249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c7c24-e2d8-49f2-add1-80ab3c14a507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:54.183889 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c7c24-e2d8-49f2-add1-80ab3c14a507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:54.185737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c7c24-e2d8-49f2-add1-80ab3c14a507 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:54.196522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1217/2420] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:38:54.200871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659e3ce-aebc-46c7-a9bd-902e23545ccb admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:54.211674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659e3ce-aebc-46c7-a9bd-902e23545ccb admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:54.217571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b61ba-0eb2-427a-8e31-86bc0514ca5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:54.218164 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b61ba-0eb2-427a-8e31-86bc0514ca5b admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:54.218438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b61ba-0eb2-427a-8e31-86bc0514ca5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:54.218716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b61ba-0eb2-427a-8e31-86bc0514ca5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:54.219355 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b61ba-0eb2-427a-8e31-86bc0514ca5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:54.221455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b61ba-0eb2-427a-8e31-86bc0514ca5b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:54.232517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1204/2421] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:54.236772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c7c24-e2d8-49f2-add1-80ab3c14a507 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:54.247117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c7c24-e2d8-49f2-add1-80ab3c14a507 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:54.252798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00895eec-fbc7-43ec-bda2-5385f1c4cc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:54.253437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00895eec-fbc7-43ec-bda2-5385f1c4cc28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:54.253738 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00895eec-fbc7-43ec-bda2-5385f1c4cc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:54.253997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00895eec-fbc7-43ec-bda2-5385f1c4cc28 admin admin] PATH_INFO: `/v3/projects/0418e5a5d0bc40449da376f2566ccc02/users/0321bcb3a6294368b87fe0b60c93a7bc/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:54.270496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00895eec-fbc7-43ec-bda2-5385f1c4cc28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0418e5a5d0bc40449da376f2566ccc02, user_id=0321bcb3a6294368b87fe0b60c93a7bc, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:54.273110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00895eec-fbc7-43ec-bda2-5385f1c4cc28 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:54.288141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1218/2422] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:54 2022] PUT /identity/v3/projects/0418e5a5d0bc40449da376f2566ccc02/users/0321bcb3a6294368b87fe0b60c93a7bc/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:54.300884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dec5cc-ffb9-4da5-aecc-f2131efaa154 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:54.301236 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dec5cc-ffb9-4da5-aecc-f2131efaa154 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:54.301618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dec5cc-ffb9-4da5-aecc-f2131efaa154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:54.341450 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f7dec5cc-ffb9-4da5-aecc-f2131efaa154 None None] MFA Rules not processed for user `0321bcb3a6294368b87fe0b60c93a7bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:54.343134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f7dec5cc-ffb9-4da5-aecc-f2131efaa154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:54.430796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1205/2423] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:54.466027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00895eec-fbc7-43ec-bda2-5385f1c4cc28 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:54.468060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-00895eec-fbc7-43ec-bda2-5385f1c4cc28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:54.527511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00895eec-fbc7-43ec-bda2-5385f1c4cc28 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:54.531815 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f188a-973b-4e29-95ad-3c47aba257b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:54.532411 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f188a-973b-4e29-95ad-3c47aba257b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:54.532698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f188a-973b-4e29-95ad-3c47aba257b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:54.532985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f188a-973b-4e29-95ad-3c47aba257b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:54.542522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f188a-973b-4e29-95ad-3c47aba257b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:54.545175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f188a-973b-4e29-95ad-3c47aba257b1 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:54.555180 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1219/2424] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:54.571186 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dec5cc-ffb9-4da5-aecc-f2131efaa154 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:54.631963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:54.638982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dec5cc-ffb9-4da5-aecc-f2131efaa154 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:54.645020 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:54.645691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:54.646022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:54.646297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] PATH_INFO: `/v3/users/0321bcb3a6294368b87fe0b60c93a7bc` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:54.647591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=0321bcb3a6294368b87fe0b60c93a7bc)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:54.649662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:54.699113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0321bcb3a6294368b87fe0b60c93a7bc'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:54.699776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0321bcb3a6294368b87fe0b60c93a7bc'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:54.700496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0321bcb3a6294368b87fe0b60c93a7bc'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:54.714728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0321bcb3a6294368b87fe0b60c93a7bc'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:54.724443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0321bcb3a6294368b87fe0b60c93a7bc'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:54.739121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1206/2425] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:54 2022] DELETE /identity/v3/users/0321bcb3a6294368b87fe0b60c93a7bc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:54.743800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e0dd0-a598-4148-911e-2cc9aa5b5ba1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:54.744116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e0dd0-a598-4148-911e-2cc9aa5b5ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:54.744501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e0dd0-a598-4148-911e-2cc9aa5b5ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:54.772810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a25e0dd0-a598-4148-911e-2cc9aa5b5ba1 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:54.774148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a25e0dd0-a598-4148-911e-2cc9aa5b5ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:54.835380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1220/2426] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:54.845393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:54.847003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:54.910261 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fb008-dab2-407c-8b22-9d1fbdb643ef admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:54.914763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2c804-2137-4aac-b50a-e90db58121e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:54.915372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2c804-2137-4aac-b50a-e90db58121e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:54.915624 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2c804-2137-4aac-b50a-e90db58121e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:54.915921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2c804-2137-4aac-b50a-e90db58121e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:54.926863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2c804-2137-4aac-b50a-e90db58121e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:54.928314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2c804-2137-4aac-b50a-e90db58121e5 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:54.938246 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1207/2427] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:55.378804 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e0dd0-a598-4148-911e-2cc9aa5b5ba1 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:55.380290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a25e0dd0-a598-4148-911e-2cc9aa5b5ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:55.393022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e0dd0-a598-4148-911e-2cc9aa5b5ba1 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:55.399799 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aff404-40d5-4533-a405-58520e7fa734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:55.400379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aff404-40d5-4533-a405-58520e7fa734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:55.400618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aff404-40d5-4533-a405-58520e7fa734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:55.400814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aff404-40d5-4533-a405-58520e7fa734 admin admin] PATH_INFO: `/v3/projects/0418e5a5d0bc40449da376f2566ccc02` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:55.402075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aff404-40d5-4533-a405-58520e7fa734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0418e5a5d0bc40449da376f2566ccc02)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:55.405671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aff404-40d5-4533-a405-58520e7fa734 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:55.435041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:55.459907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-91aff404-40d5-4533-a405-58520e7fa734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0418e5a5d0bc40449da376f2566ccc02', 'request_id': 'req-91aff404-40d5-4533-a405-58520e7fa734'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:55.482258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-91aff404-40d5-4533-a405-58520e7fa734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0418e5a5d0bc40449da376f2566ccc02', 'request_id': 'req-91aff404-40d5-4533-a405-58520e7fa734'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:55.489071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-91aff404-40d5-4533-a405-58520e7fa734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0418e5a5d0bc40449da376f2566ccc02', 'request_id': 'req-91aff404-40d5-4533-a405-58520e7fa734'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:55.500987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-91aff404-40d5-4533-a405-58520e7fa734 admin admin] The token cache is being invalidate because project 0418e5a5d0bc40449da376f2566ccc02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:55.501269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-91aff404-40d5-4533-a405-58520e7fa734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:55.501935 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-91aff404-40d5-4533-a405-58520e7fa734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:55.503943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1221/2428] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:55 2022] DELETE /identity/v3/projects/0418e5a5d0bc40449da376f2566ccc02 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:55.508864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e5f72-ea5e-4ba6-aac8-b56a50330e8a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:55.509151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e5f72-ea5e-4ba6-aac8-b56a50330e8a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:55.509453 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e5f72-ea5e-4ba6-aac8-b56a50330e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:55.529467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d45e5f72-ea5e-4ba6-aac8-b56a50330e8a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:55.530674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d45e5f72-ea5e-4ba6-aac8-b56a50330e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:55.594847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1208/2429] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:55.605610 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0c372-4305-45be-bb45-4adce33bfcf6 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:55.605899 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0c372-4305-45be-bb45-4adce33bfcf6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:55.606162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0c372-4305-45be-bb45-4adce33bfcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:55.627438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-53f0c372-4305-45be-bb45-4adce33bfcf6 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:55.628704 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-53f0c372-4305-45be-bb45-4adce33bfcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:55.637841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1222/2430] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:55.642392 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e5f72-ea5e-4ba6-aac8-b56a50330e8a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:55.652296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e5f72-ea5e-4ba6-aac8-b56a50330e8a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:55.657572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baee51e-7dcb-4614-8e65-e7f8affcbf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:55.658145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baee51e-7dcb-4614-8e65-e7f8affcbf41 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:55.658416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baee51e-7dcb-4614-8e65-e7f8affcbf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:55.658681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baee51e-7dcb-4614-8e65-e7f8affcbf41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:55.659325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baee51e-7dcb-4614-8e65-e7f8affcbf41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:38:55.659621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baee51e-7dcb-4614-8e65-e7f8affcbf41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:55.661270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baee51e-7dcb-4614-8e65-e7f8affcbf41 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:55.678205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1209/2431] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:38:55.682627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0c372-4305-45be-bb45-4adce33bfcf6 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:55.692394 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0c372-4305-45be-bb45-4adce33bfcf6 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:55.697623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc4224-dd4c-4dd2-9885-d72c203d9d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:55.698220 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc4224-dd4c-4dd2-9885-d72c203d9d00 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:55.698499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc4224-dd4c-4dd2-9885-d72c203d9d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:55.698767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc4224-dd4c-4dd2-9885-d72c203d9d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:55.699387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc4224-dd4c-4dd2-9885-d72c203d9d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:55.701526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc4224-dd4c-4dd2-9885-d72c203d9d00 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:55.730538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1223/2432] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:55 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:55.734889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baee51e-7dcb-4614-8e65-e7f8affcbf41 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:55.741454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:55.745825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baee51e-7dcb-4614-8e65-e7f8affcbf41 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:55.805122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57dc6e-2ce7-4552-a3e0-d0bd077eff01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:55.805716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57dc6e-2ce7-4552-a3e0-d0bd077eff01 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:55.805989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57dc6e-2ce7-4552-a3e0-d0bd077eff01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:55.806257 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57dc6e-2ce7-4552-a3e0-d0bd077eff01 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:55.806877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57dc6e-2ce7-4552-a3e0-d0bd077eff01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:55.808835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57dc6e-2ce7-4552-a3e0-d0bd077eff01 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:55.833036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1210/2433] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:55 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:55.837115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc4224-dd4c-4dd2-9885-d72c203d9d00 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:55.848332 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc4224-dd4c-4dd2-9885-d72c203d9d00 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:55.853965 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdda9d-0e15-46ea-8ec2-588265210da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:55.854574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdda9d-0e15-46ea-8ec2-588265210da7 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:55.854852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdda9d-0e15-46ea-8ec2-588265210da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:55.855121 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdda9d-0e15-46ea-8ec2-588265210da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:55.855765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdda9d-0e15-46ea-8ec2-588265210da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:55.857578 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdda9d-0e15-46ea-8ec2-588265210da7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:55.868302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1224/2434] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:55.872947 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57dc6e-2ce7-4552-a3e0-d0bd077eff01 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:55.883271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57dc6e-2ce7-4552-a3e0-d0bd077eff01 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:55.888894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24730c-1f2c-4853-a2a6-b52124679ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:55.889245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24730c-1f2c-4853-a2a6-b52124679ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:55.889521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24730c-1f2c-4853-a2a6-b52124679ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:55.889791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24730c-1f2c-4853-a2a6-b52124679ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:55.890434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24730c-1f2c-4853-a2a6-b52124679ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:55.892113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24730c-1f2c-4853-a2a6-b52124679ce1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:55.902908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1211/2435] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:55.907232 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdda9d-0e15-46ea-8ec2-588265210da7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:55.917799 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdda9d-0e15-46ea-8ec2-588265210da7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:55.924033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bc055-880f-4364-9c5d-3d770480d64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:55.924684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bc055-880f-4364-9c5d-3d770480d64d admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:55.924987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bc055-880f-4364-9c5d-3d770480d64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:55.925256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bc055-880f-4364-9c5d-3d770480d64d admin admin] PATH_INFO: `/v3/projects/befa66a734b44ec08956d478f5213031/users/47e88b1b9d7b4753a35ca43eabc04a4c/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:55.941178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bc055-880f-4364-9c5d-3d770480d64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=befa66a734b44ec08956d478f5213031, user_id=47e88b1b9d7b4753a35ca43eabc04a4c, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:55.943893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bc055-880f-4364-9c5d-3d770480d64d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:55.959007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1225/2436] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:55 2022] PUT /identity/v3/projects/befa66a734b44ec08956d478f5213031/users/47e88b1b9d7b4753a35ca43eabc04a4c/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:55.971099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2b96d-f718-4349-9500-b7a5a99d37c1 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:55.971380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2b96d-f718-4349-9500-b7a5a99d37c1 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:55.971654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2b96d-f718-4349-9500-b7a5a99d37c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.002011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5ac2b96d-f718-4349-9500-b7a5a99d37c1 None None] MFA Rules not processed for user `47e88b1b9d7b4753a35ca43eabc04a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:56.003227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5ac2b96d-f718-4349-9500-b7a5a99d37c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:56.073455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1212/2437] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:56.078387 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bc055-880f-4364-9c5d-3d770480d64d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:56.143160 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bc055-880f-4364-9c5d-3d770480d64d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:56.148910 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c898c4-34a8-4ad6-98de-fa7c4ea3e61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:56.149499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c898c4-34a8-4ad6-98de-fa7c4ea3e61b admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.149773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c898c4-34a8-4ad6-98de-fa7c4ea3e61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.150038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c898c4-34a8-4ad6-98de-fa7c4ea3e61b admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.150644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c898c4-34a8-4ad6-98de-fa7c4ea3e61b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:56.152570 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c898c4-34a8-4ad6-98de-fa7c4ea3e61b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:56.185545 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1226/2438] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:56 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:56.191233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2b96d-f718-4349-9500-b7a5a99d37c1 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:56.203237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2b96d-f718-4349-9500-b7a5a99d37c1 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:56.262038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aea931-d846-4750-892e-af0c6127399c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:56.262651 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aea931-d846-4750-892e-af0c6127399c admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.262929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aea931-d846-4750-892e-af0c6127399c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.263808 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aea931-d846-4750-892e-af0c6127399c admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.264315 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aea931-d846-4750-892e-af0c6127399c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:56.266301 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aea931-d846-4750-892e-af0c6127399c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:56.288798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1213/2439] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:56 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:56.292687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c898c4-34a8-4ad6-98de-fa7c4ea3e61b admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:56.303834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c898c4-34a8-4ad6-98de-fa7c4ea3e61b admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:56.309219 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e910202-01f3-425e-8724-2d2824b49b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:56.309882 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e910202-01f3-425e-8724-2d2824b49b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.310241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e910202-01f3-425e-8724-2d2824b49b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.310593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e910202-01f3-425e-8724-2d2824b49b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.311643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e910202-01f3-425e-8724-2d2824b49b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:56.313308 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e910202-01f3-425e-8724-2d2824b49b0b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:56.326418 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1227/2440] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:56.331101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aea931-d846-4750-892e-af0c6127399c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:56.339851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:56.347881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aea931-d846-4750-892e-af0c6127399c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:56.355575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c150f4-0712-44bc-8512-b034e1d7045d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:56.356332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c150f4-0712-44bc-8512-b034e1d7045d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.356607 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c150f4-0712-44bc-8512-b034e1d7045d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.356960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c150f4-0712-44bc-8512-b034e1d7045d admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.357578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c150f4-0712-44bc-8512-b034e1d7045d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:56.359245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c150f4-0712-44bc-8512-b034e1d7045d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:56.370481 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1214/2441] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:56.375290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e910202-01f3-425e-8724-2d2824b49b0b admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:56.387155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e910202-01f3-425e-8724-2d2824b49b0b admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:56.392194 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db17a4-c9dd-48c1-8df4-02658da338e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:56.392969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db17a4-c9dd-48c1-8df4-02658da338e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.393627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db17a4-c9dd-48c1-8df4-02658da338e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.393627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db17a4-c9dd-48c1-8df4-02658da338e6 admin admin] PATH_INFO: `/v3/projects/14ab406c2e3347ca8615a093fac84985/users/f408df2183c14315bda270c61aed9407/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.409230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db17a4-c9dd-48c1-8df4-02658da338e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14ab406c2e3347ca8615a093fac84985, user_id=f408df2183c14315bda270c61aed9407, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:56.412064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db17a4-c9dd-48c1-8df4-02658da338e6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:56.424542 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:56.430807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1228/2442] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:56 2022] PUT /identity/v3/projects/14ab406c2e3347ca8615a093fac84985/users/f408df2183c14315bda270c61aed9407/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:56.441629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e0430-e6bc-4b8d-a7c7-0a783b978fff None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.441974 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e0430-e6bc-4b8d-a7c7-0a783b978fff None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.442218 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e0430-e6bc-4b8d-a7c7-0a783b978fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.476064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-889e0430-e6bc-4b8d-a7c7-0a783b978fff None None] MFA Rules not processed for user `f408df2183c14315bda270c61aed9407`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:56.477348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-889e0430-e6bc-4b8d-a7c7-0a783b978fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:56.549542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1215/2443] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:56.554177 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db17a4-c9dd-48c1-8df4-02658da338e6 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:56.619092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db17a4-c9dd-48c1-8df4-02658da338e6 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:56.624137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce95ec6-910a-4d1e-9a40-9da0df14a4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:56.624772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce95ec6-910a-4d1e-9a40-9da0df14a4b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.625046 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce95ec6-910a-4d1e-9a40-9da0df14a4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.625314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce95ec6-910a-4d1e-9a40-9da0df14a4b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.625908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce95ec6-910a-4d1e-9a40-9da0df14a4b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:56.627839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce95ec6-910a-4d1e-9a40-9da0df14a4b2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:56.656971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1229/2444] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:56 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:38:56.661246 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e0430-e6bc-4b8d-a7c7-0a783b978fff None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:56.675743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e0430-e6bc-4b8d-a7c7-0a783b978fff None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:56.741565 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac5aa0-5fc2-4ae9-926c-414ad74a2a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:56.742176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac5aa0-5fc2-4ae9-926c-414ad74a2a2a admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.742457 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac5aa0-5fc2-4ae9-926c-414ad74a2a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.742758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac5aa0-5fc2-4ae9-926c-414ad74a2a2a admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.743383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac5aa0-5fc2-4ae9-926c-414ad74a2a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:56.746324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac5aa0-5fc2-4ae9-926c-414ad74a2a2a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:56.771904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1216/2445] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:56 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:38:56.773258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce95ec6-910a-4d1e-9a40-9da0df14a4b2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:56.785356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce95ec6-910a-4d1e-9a40-9da0df14a4b2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:56.791020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f972bb1-a8e1-452e-8021-b669eea73e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:56.791626 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f972bb1-a8e1-452e-8021-b669eea73e44 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.791934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f972bb1-a8e1-452e-8021-b669eea73e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.792199 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f972bb1-a8e1-452e-8021-b669eea73e44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.792913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f972bb1-a8e1-452e-8021-b669eea73e44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:56.794712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f972bb1-a8e1-452e-8021-b669eea73e44 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:56.808523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1230/2446] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:38:56.813588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac5aa0-5fc2-4ae9-926c-414ad74a2a2a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:56.823852 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac5aa0-5fc2-4ae9-926c-414ad74a2a2a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:56.828708 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473f11-a0d9-4091-ada8-d9a19ce69b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:56.829370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473f11-a0d9-4091-ada8-d9a19ce69b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.829646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473f11-a0d9-4091-ada8-d9a19ce69b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.829920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b473f11-a0d9-4091-ada8-d9a19ce69b2a admin admin] PATH_INFO: `/v3/projects/7aa18e7910ca4e0095719df5efc5ca18/users/b20883d7eb7741c2b4563da94102bf93/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.845839 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b473f11-a0d9-4091-ada8-d9a19ce69b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa18e7910ca4e0095719df5efc5ca18, user_id=b20883d7eb7741c2b4563da94102bf93, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:56.848513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b473f11-a0d9-4091-ada8-d9a19ce69b2a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:56.863079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1217/2447] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:56 2022] PUT /identity/v3/projects/7aa18e7910ca4e0095719df5efc5ca18/users/b20883d7eb7741c2b4563da94102bf93/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:56.867178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f972bb1-a8e1-452e-8021-b669eea73e44 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:56.878448 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f972bb1-a8e1-452e-8021-b669eea73e44 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:56.939584 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c515d0-bcbe-49e5-b429-12f30be177d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:56.940104 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c515d0-bcbe-49e5-b429-12f30be177d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.940375 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c515d0-bcbe-49e5-b429-12f30be177d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.940636 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c515d0-bcbe-49e5-b429-12f30be177d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.941304 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c515d0-bcbe-49e5-b429-12f30be177d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:56.942924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c515d0-bcbe-49e5-b429-12f30be177d2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:56.953197 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1231/2448] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:56.957682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473f11-a0d9-4091-ada8-d9a19ce69b2a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:56.968557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b473f11-a0d9-4091-ada8-d9a19ce69b2a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:56.974323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b55421-808f-4f3b-bc4d-bbc871d2a47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:56.975010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b55421-808f-4f3b-bc4d-bbc871d2a47e admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.975285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b55421-808f-4f3b-bc4d-bbc871d2a47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.975553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b55421-808f-4f3b-bc4d-bbc871d2a47e admin admin] PATH_INFO: `/v3/projects/7aa18e7910ca4e0095719df5efc5ca18/users/b20883d7eb7741c2b4563da94102bf93/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.977676 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b55421-808f-4f3b-bc4d-bbc871d2a47e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa18e7910ca4e0095719df5efc5ca18, user_id=b20883d7eb7741c2b4563da94102bf93, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:56.980734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b55421-808f-4f3b-bc4d-bbc871d2a47e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:56.992434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1218/2449] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:56 2022] PUT /identity/v3/projects/7aa18e7910ca4e0095719df5efc5ca18/users/b20883d7eb7741c2b4563da94102bf93/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:57.003287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770d061-c3d1-49fb-94f9-176ee9c3131b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:57.003581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770d061-c3d1-49fb-94f9-176ee9c3131b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:57.003894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770d061-c3d1-49fb-94f9-176ee9c3131b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:57.033384 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7770d061-c3d1-49fb-94f9-176ee9c3131b None None] MFA Rules not processed for user `b20883d7eb7741c2b4563da94102bf93`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:57.034597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7770d061-c3d1-49fb-94f9-176ee9c3131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:57.103958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:57.116096 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1232/2450] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:38:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:57.128041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b55421-808f-4f3b-bc4d-bbc871d2a47e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:57.130082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-52b55421-808f-4f3b-bc4d-bbc871d2a47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:57.173899 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:57.199151 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b55421-808f-4f3b-bc4d-bbc871d2a47e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:57.203173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e0ecd-f060-4782-bd5c-f273b154284d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:57.203770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e0ecd-f060-4782-bd5c-f273b154284d service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:57.204048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e0ecd-f060-4782-bd5c-f273b154284d service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:57.204314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e0ecd-f060-4782-bd5c-f273b154284d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:57.263273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e0ecd-f060-4782-bd5c-f273b154284d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:57.264851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e0ecd-f060-4782-bd5c-f273b154284d service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:57.274455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1219/2451] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:38:58.009239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770d061-c3d1-49fb-94f9-176ee9c3131b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:58.010842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7770d061-c3d1-49fb-94f9-176ee9c3131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:58.076752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770d061-c3d1-49fb-94f9-176ee9c3131b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:58.080620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb781f0-14e5-454a-b428-907dc0ed2b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:58.081262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb781f0-14e5-454a-b428-907dc0ed2b9a service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:58.081539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb781f0-14e5-454a-b428-907dc0ed2b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:58.081808 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb781f0-14e5-454a-b428-907dc0ed2b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:58.083694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dcb781f0-14e5-454a-b428-907dc0ed2b9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:58.156299 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb781f0-14e5-454a-b428-907dc0ed2b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:58.157795 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb781f0-14e5-454a-b428-907dc0ed2b9a service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:58.167967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1233/2452] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:58.805517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e0ecd-f060-4782-bd5c-f273b154284d service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:58.807316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d90e0ecd-f060-4782-bd5c-f273b154284d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:58.821160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e0ecd-f060-4782-bd5c-f273b154284d service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:58.826672 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28395b1-4a27-4154-bfb4-21ebd0d7da56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:58.827388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28395b1-4a27-4154-bfb4-21ebd0d7da56 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:58.827668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28395b1-4a27-4154-bfb4-21ebd0d7da56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:58.827937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28395b1-4a27-4154-bfb4-21ebd0d7da56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:58.891455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28395b1-4a27-4154-bfb4-21ebd0d7da56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:58.892962 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28395b1-4a27-4154-bfb4-21ebd0d7da56 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:58.902559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1220/2453] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:59.494404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb781f0-14e5-454a-b428-907dc0ed2b9a service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:59.504892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb781f0-14e5-454a-b428-907dc0ed2b9a service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:59.509208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48e746-f7ad-42e0-8c1c-7766a6d22081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:59.509818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48e746-f7ad-42e0-8c1c-7766a6d22081 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:59.510260 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48e746-f7ad-42e0-8c1c-7766a6d22081 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:59.510543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48e746-f7ad-42e0-8c1c-7766a6d22081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:59.522437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48e746-f7ad-42e0-8c1c-7766a6d22081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:59.523871 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48e746-f7ad-42e0-8c1c-7766a6d22081 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:59.534763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1234/2454] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:01.220631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4e22d-9176-4cbe-8d74-c511d4528450 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:01.221112 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4e22d-9176-4cbe-8d74-c511d4528450 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:01.221220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4e22d-9176-4cbe-8d74-c511d4528450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:01.241621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-0df4e22d-9176-4cbe-8d74-c511d4528450 None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:01.242876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0df4e22d-9176-4cbe-8d74-c511d4528450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:01.302910 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1221/2455] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:39:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:01.355324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48e746-f7ad-42e0-8c1c-7766a6d22081 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:01.356944 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dd48e746-f7ad-42e0-8c1c-7766a6d22081 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:01.431175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48e746-f7ad-42e0-8c1c-7766a6d22081 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:01.437146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b9d0a-2ccf-4972-ad42-b5034c83e52f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:01.437764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b9d0a-2ccf-4972-ad42-b5034c83e52f service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:01.438043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b9d0a-2ccf-4972-ad42-b5034c83e52f service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:01.438427 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b9d0a-2ccf-4972-ad42-b5034c83e52f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:01.447013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b9d0a-2ccf-4972-ad42-b5034c83e52f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:01.448595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b9d0a-2ccf-4972-ad42-b5034c83e52f service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:01.459139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1235/2456] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:39:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:39:03.626661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4e22d-9176-4cbe-8d74-c511d4528450 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:03.664516 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:03.705262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4e22d-9176-4cbe-8d74-c511d4528450 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:03.711153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:03.712178 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:03.712668 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:03.713147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] PATH_INFO: `/v3/users/47e88b1b9d7b4753a35ca43eabc04a4c` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:03.715406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] RBAC: Authorizing `identity:delete_user(user_id=47e88b1b9d7b4753a35ca43eabc04a4c)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:03.718820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:03.783833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47e88b1b9d7b4753a35ca43eabc04a4c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:03.784876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47e88b1b9d7b4753a35ca43eabc04a4c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:03.786050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47e88b1b9d7b4753a35ca43eabc04a4c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:03.807390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47e88b1b9d7b4753a35ca43eabc04a4c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:03.821461 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47e88b1b9d7b4753a35ca43eabc04a4c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:03.843324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1222/2457] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:03 2022] DELETE /identity/v3/users/47e88b1b9d7b4753a35ca43eabc04a4c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:03.847337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f8c4c-4b8a-4fb7-9de3-cadff0b8b75e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:03.847745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f8c4c-4b8a-4fb7-9de3-cadff0b8b75e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:03.847859 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f8c4c-4b8a-4fb7-9de3-cadff0b8b75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:03.877517 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-864f8c4c-4b8a-4fb7-9de3-cadff0b8b75e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:03.879320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-864f8c4c-4b8a-4fb7-9de3-cadff0b8b75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:03.901242 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:03.995670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1236/2458] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:04.006000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:04.007510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:04.077945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fc6fd-d16a-43f8-ab82-07a966fec26f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:04.084891 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ec01e-ac83-4cfb-b57d-6ac8e6dfbe6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:04.085517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ec01e-ac83-4cfb-b57d-6ac8e6dfbe6a service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:04.085994 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ec01e-ac83-4cfb-b57d-6ac8e6dfbe6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:04.086268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ec01e-ac83-4cfb-b57d-6ac8e6dfbe6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:04.096385 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ec01e-ac83-4cfb-b57d-6ac8e6dfbe6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:04.098172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ec01e-ac83-4cfb-b57d-6ac8e6dfbe6a service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:04.112051 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1223/2459] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:04.413140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f8c4c-4b8a-4fb7-9de3-cadff0b8b75e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:04.414809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-864f8c4c-4b8a-4fb7-9de3-cadff0b8b75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:04.431303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f8c4c-4b8a-4fb7-9de3-cadff0b8b75e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:04.436805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:04.439104 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:04.439828 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:04.440279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] PATH_INFO: `/v3/projects/befa66a734b44ec08956d478f5213031` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:04.442339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] RBAC: Authorizing `identity:delete_project(project_id=befa66a734b44ec08956d478f5213031)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:04.444904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:04.511709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'befa66a734b44ec08956d478f5213031', 'request_id': 'req-21ff4685-3a7b-44d9-87a0-33d1493094af'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:04.533545 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'befa66a734b44ec08956d478f5213031', 'request_id': 'req-21ff4685-3a7b-44d9-87a0-33d1493094af'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:04.544924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'befa66a734b44ec08956d478f5213031', 'request_id': 'req-21ff4685-3a7b-44d9-87a0-33d1493094af'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:04.561114 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] The token cache is being invalidate because project befa66a734b44ec08956d478f5213031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:04.561831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:04.562669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:04.567264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1237/2460] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:04 2022] DELETE /identity/v3/projects/befa66a734b44ec08956d478f5213031 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:04.568329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ec01e-ac83-4cfb-b57d-6ac8e6dfbe6a service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:04.578727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ec01e-ac83-4cfb-b57d-6ac8e6dfbe6a service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:04.641732 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:04.659548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:04.660103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:04.660381 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:04.660746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] PATH_INFO: `/v3/users/f408df2183c14315bda270c61aed9407` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:04.663728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f408df2183c14315bda270c61aed9407)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:04.663905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:04.734567 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f408df2183c14315bda270c61aed9407'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:04.735440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f408df2183c14315bda270c61aed9407'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:04.736021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f408df2183c14315bda270c61aed9407'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:04.764310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f408df2183c14315bda270c61aed9407'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:04.766407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f408df2183c14315bda270c61aed9407'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:04.782573 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1224/2461] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:04 2022] DELETE /identity/v3/users/f408df2183c14315bda270c61aed9407 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:05.211378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:05.212870 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:05.244515 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff4685-3a7b-44d9-87a0-33d1493094af admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:05.336382 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:05.337188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:05.337560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:05.337943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] PATH_INFO: `/v3/projects/14ab406c2e3347ca8615a093fac84985` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:05.339148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=14ab406c2e3347ca8615a093fac84985)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:05.341382 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:05.399758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:05.403012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14ab406c2e3347ca8615a093fac84985', 'request_id': 'req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:05.412867 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14ab406c2e3347ca8615a093fac84985', 'request_id': 'req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:05.421053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14ab406c2e3347ca8615a093fac84985', 'request_id': 'req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:05.433268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] The token cache is being invalidate because project 14ab406c2e3347ca8615a093fac84985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:05.433755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:05.434475 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:05.436443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1238/2462] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:05 2022] DELETE /identity/v3/projects/14ab406c2e3347ca8615a093fac84985 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:05.440599 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:05.442189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:05.457057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157b8c8-8065-4ccb-8384-ac6d08555b7c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:05.534325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61e257-bcac-4a93-affb-9f2308515ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:05.535103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61e257-bcac-4a93-affb-9f2308515ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:05.535463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61e257-bcac-4a93-affb-9f2308515ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:05.535752 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61e257-bcac-4a93-affb-9f2308515ee0 admin admin] PATH_INFO: `/v3/users/b20883d7eb7741c2b4563da94102bf93` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:05.537135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61e257-bcac-4a93-affb-9f2308515ee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b20883d7eb7741c2b4563da94102bf93)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:05.539167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61e257-bcac-4a93-affb-9f2308515ee0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:05.592111 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3a61e257-bcac-4a93-affb-9f2308515ee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b20883d7eb7741c2b4563da94102bf93'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:05.592885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3a61e257-bcac-4a93-affb-9f2308515ee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b20883d7eb7741c2b4563da94102bf93'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:05.593547 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3a61e257-bcac-4a93-affb-9f2308515ee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b20883d7eb7741c2b4563da94102bf93'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:05.606857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3a61e257-bcac-4a93-affb-9f2308515ee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b20883d7eb7741c2b4563da94102bf93'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:05.617092 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-3a61e257-bcac-4a93-affb-9f2308515ee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b20883d7eb7741c2b4563da94102bf93'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:05.622026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:05.638795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1225/2463] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:05 2022] DELETE /identity/v3/users/b20883d7eb7741c2b4563da94102bf93 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:06.013279 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:06.015538 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:06.038066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7947f-8afa-4eaf-926b-86d7ef3e07ff admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:06.120354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07932485-ed4f-4519-ab22-aae480023908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:06.121196 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07932485-ed4f-4519-ab22-aae480023908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:06.121549 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07932485-ed4f-4519-ab22-aae480023908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:06.122009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07932485-ed4f-4519-ab22-aae480023908 admin admin] PATH_INFO: `/v3/projects/7aa18e7910ca4e0095719df5efc5ca18` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:06.123883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07932485-ed4f-4519-ab22-aae480023908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa18e7910ca4e0095719df5efc5ca18)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:06.126296 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07932485-ed4f-4519-ab22-aae480023908 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:06.186833 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-07932485-ed4f-4519-ab22-aae480023908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa18e7910ca4e0095719df5efc5ca18', 'request_id': 'req-07932485-ed4f-4519-ab22-aae480023908'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:06.202559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-07932485-ed4f-4519-ab22-aae480023908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa18e7910ca4e0095719df5efc5ca18', 'request_id': 'req-07932485-ed4f-4519-ab22-aae480023908'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:06.210300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-07932485-ed4f-4519-ab22-aae480023908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa18e7910ca4e0095719df5efc5ca18', 'request_id': 'req-07932485-ed4f-4519-ab22-aae480023908'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:06.227362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-07932485-ed4f-4519-ab22-aae480023908 admin admin] The token cache is being invalidate because project 7aa18e7910ca4e0095719df5efc5ca18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:06.228032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-07932485-ed4f-4519-ab22-aae480023908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:06.228312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-07932485-ed4f-4519-ab22-aae480023908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:06.230368 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1239/2464] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:06 2022] DELETE /identity/v3/projects/7aa18e7910ca4e0095719df5efc5ca18 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:06.237223 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e36766-1cf3-4391-952e-4fc5a566b84b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:06.238077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e36766-1cf3-4391-952e-4fc5a566b84b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:06.238541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e36766-1cf3-4391-952e-4fc5a566b84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:06.264503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-82e36766-1cf3-4391-952e-4fc5a566b84b None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:06.266014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-82e36766-1cf3-4391-952e-4fc5a566b84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:06.344784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1226/2465] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:06.415217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d390f-e68d-43c7-8246-f7f9a6ef5e74 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:06.415539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d390f-e68d-43c7-8246-f7f9a6ef5e74 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:06.415759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d390f-e68d-43c7-8246-f7f9a6ef5e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:06.440371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-fb6d390f-e68d-43c7-8246-f7f9a6ef5e74 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:06.441613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fb6d390f-e68d-43c7-8246-f7f9a6ef5e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:06.451050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1240/2466] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:06.455381 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e36766-1cf3-4391-952e-4fc5a566b84b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:06.469369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e36766-1cf3-4391-952e-4fc5a566b84b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:06.476071 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf92be7-06e0-474b-8830-a45a9752f07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:06.477150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf92be7-06e0-474b-8830-a45a9752f07e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:06.477634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf92be7-06e0-474b-8830-a45a9752f07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:06.477949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf92be7-06e0-474b-8830-a45a9752f07e admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:06.478984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf92be7-06e0-474b-8830-a45a9752f07e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:39:06.479295 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf92be7-06e0-474b-8830-a45a9752f07e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:06.481279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf92be7-06e0-474b-8830-a45a9752f07e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:06.496682 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1227/2467] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:39:06.499586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d390f-e68d-43c7-8246-f7f9a6ef5e74 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:06.513228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d390f-e68d-43c7-8246-f7f9a6ef5e74 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:06.518697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8e7ea-fcef-4d52-aa6c-3d169844bc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:06.519423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8e7ea-fcef-4d52-aa6c-3d169844bc46 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:06.519923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8e7ea-fcef-4d52-aa6c-3d169844bc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:06.520331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8e7ea-fcef-4d52-aa6c-3d169844bc46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:06.521042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8e7ea-fcef-4d52-aa6c-3d169844bc46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:06.523047 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8e7ea-fcef-4d52-aa6c-3d169844bc46 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:06.572833 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1241/2468] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:06 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:06.578275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf92be7-06e0-474b-8830-a45a9752f07e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:06.589834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf92be7-06e0-474b-8830-a45a9752f07e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:06.660099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0fdc3-e931-4e4b-bd6d-7881e5d19d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:06.664044 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0fdc3-e931-4e4b-bd6d-7881e5d19d50 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:06.664484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0fdc3-e931-4e4b-bd6d-7881e5d19d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:06.664876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0fdc3-e931-4e4b-bd6d-7881e5d19d50 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:06.665770 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0fdc3-e931-4e4b-bd6d-7881e5d19d50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:06.667834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0fdc3-e931-4e4b-bd6d-7881e5d19d50 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:06.693530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1228/2469] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:06 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:06.697670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8e7ea-fcef-4d52-aa6c-3d169844bc46 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:06.709511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8e7ea-fcef-4d52-aa6c-3d169844bc46 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:06.714581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9f89f-622a-4749-9d4a-ddcb6148b8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:06.715370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9f89f-622a-4749-9d4a-ddcb6148b8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:06.716297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9f89f-622a-4749-9d4a-ddcb6148b8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:06.716754 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9f89f-622a-4749-9d4a-ddcb6148b8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:06.717548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9f89f-622a-4749-9d4a-ddcb6148b8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:06.719365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9f89f-622a-4749-9d4a-ddcb6148b8f5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:06.731066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1242/2470] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:06.738245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0fdc3-e931-4e4b-bd6d-7881e5d19d50 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:06.749257 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0fdc3-e931-4e4b-bd6d-7881e5d19d50 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:06.757050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bcdeb-96b3-4e1b-b5c9-23f59ea8f10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:06.757797 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bcdeb-96b3-4e1b-b5c9-23f59ea8f10a admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:06.758185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bcdeb-96b3-4e1b-b5c9-23f59ea8f10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:06.758530 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bcdeb-96b3-4e1b-b5c9-23f59ea8f10a admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:06.759347 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bcdeb-96b3-4e1b-b5c9-23f59ea8f10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:06.761003 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bcdeb-96b3-4e1b-b5c9-23f59ea8f10a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:06.776809 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1229/2471] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:06.779972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9f89f-622a-4749-9d4a-ddcb6148b8f5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:06.790857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9f89f-622a-4749-9d4a-ddcb6148b8f5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:06.799185 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf8356-468a-4337-8068-8afb9685373b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:06.799941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf8356-468a-4337-8068-8afb9685373b admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:06.800283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf8356-468a-4337-8068-8afb9685373b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:06.800780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf8356-468a-4337-8068-8afb9685373b admin admin] PATH_INFO: `/v3/projects/36393e8f816345a9b86d380bad8c0e55/users/889fbdaae5a440b3b0055146bc4cfd33/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:06.824580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf8356-468a-4337-8068-8afb9685373b admin admin] RBAC: Authorizing `identity:create_grant(project_id=36393e8f816345a9b86d380bad8c0e55, user_id=889fbdaae5a440b3b0055146bc4cfd33, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:06.827571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf8356-468a-4337-8068-8afb9685373b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:06.841617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:06.847372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1243/2472] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:06 2022] PUT /identity/v3/projects/36393e8f816345a9b86d380bad8c0e55/users/889fbdaae5a440b3b0055146bc4cfd33/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:06.861669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5a947-78bf-4f60-b6f8-0d245a19aeef None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:06.864117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5a947-78bf-4f60-b6f8-0d245a19aeef None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:06.864405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5a947-78bf-4f60-b6f8-0d245a19aeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:06.905398 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ddb5a947-78bf-4f60-b6f8-0d245a19aeef None None] MFA Rules not processed for user `889fbdaae5a440b3b0055146bc4cfd33`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:06.906678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ddb5a947-78bf-4f60-b6f8-0d245a19aeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:06.964371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:06.991845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1230/2473] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:39:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:07.005619 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf8356-468a-4337-8068-8afb9685373b admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:07.008345 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cbaf8356-468a-4337-8068-8afb9685373b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:07.088414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf8356-468a-4337-8068-8afb9685373b admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:07.092788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488d9c9-c4e2-4d8a-a14a-6736d4ca3fc5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94bfed415d24a9cb596b1fa92d5df97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:07.093566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488d9c9-c4e2-4d8a-a14a-6736d4ca3fc5 service glance] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:07.093827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488d9c9-c4e2-4d8a-a14a-6736d4ca3fc5 service glance] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:07.093993 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488d9c9-c4e2-4d8a-a14a-6736d4ca3fc5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:07.104226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488d9c9-c4e2-4d8a-a14a-6736d4ca3fc5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:07.105881 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488d9c9-c4e2-4d8a-a14a-6736d4ca3fc5 service glance] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:07.116337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1244/2474] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:15.992815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ce1aa-d38b-495a-8d1f-54bab830188b None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:15.993432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ce1aa-d38b-495a-8d1f-54bab830188b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:15.993432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ce1aa-d38b-495a-8d1f-54bab830188b None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:15.994876 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1231/2475] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:39:16.005120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a2cf7-4400-48dc-af17-3bc7fecf974d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.005712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a2cf7-4400-48dc-af17-3bc7fecf974d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.005878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a2cf7-4400-48dc-af17-3bc7fecf974d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.026731 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f40a2cf7-4400-48dc-af17-3bc7fecf974d None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:16.028585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f40a2cf7-4400-48dc-af17-3bc7fecf974d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:16.113489 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1245/2476] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:39:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:16.127955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ce1aa-d38b-495a-8d1f-54bab830188b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:16.129368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-266ce1aa-d38b-495a-8d1f-54bab830188b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:16.149189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ce1aa-d38b-495a-8d1f-54bab830188b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:16.152745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c5de5-72ec-400e-b664-552ed3beee03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94bfed415d24a9cb596b1fa92d5df97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:16.153433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c5de5-72ec-400e-b664-552ed3beee03 service glance] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.153709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c5de5-72ec-400e-b664-552ed3beee03 service glance] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.153980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c5de5-72ec-400e-b664-552ed3beee03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.164898 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c5de5-72ec-400e-b664-552ed3beee03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:16.166341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c5de5-72ec-400e-b664-552ed3beee03 service glance] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:16.177396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1232/2477] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:39:16.266712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c507e7-a563-4689-aa5e-703410aa14bb None None] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.267051 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c507e7-a563-4689-aa5e-703410aa14bb None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.267351 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c507e7-a563-4689-aa5e-703410aa14bb None None] PATH_INFO: `/` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.268888 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1246/2478] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:39:16.273807 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebf95d-c735-4555-b9a9-d08983e1c72e None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.274104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebf95d-c735-4555-b9a9-d08983e1c72e None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.274371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebf95d-c735-4555-b9a9-d08983e1c72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.295685 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-75ebf95d-c735-4555-b9a9-d08983e1c72e None None] MFA Rules not processed for user `5e18461c8cc4497eae7a83d9ad1448f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:16.296939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-75ebf95d-c735-4555-b9a9-d08983e1c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:16.359201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1233/2479] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:39:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:16.371447 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c507e7-a563-4689-aa5e-703410aa14bb None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:16.373367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f4c507e7-a563-4689-aa5e-703410aa14bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:16.390040 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c507e7-a563-4689-aa5e-703410aa14bb None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:16.394763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e743537-bb6d-41c2-9c5f-58f01b57a63e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:16.395358 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e743537-bb6d-41c2-9c5f-58f01b57a63e service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.395661 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e743537-bb6d-41c2-9c5f-58f01b57a63e service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.395931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e743537-bb6d-41c2-9c5f-58f01b57a63e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.404236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e743537-bb6d-41c2-9c5f-58f01b57a63e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:16.405728 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e743537-bb6d-41c2-9c5f-58f01b57a63e service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:16.415108 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1247/2480] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:16.508845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab767338-ac0e-477c-b869-0722f6e25b1c None None] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.509185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab767338-ac0e-477c-b869-0722f6e25b1c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.509480 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab767338-ac0e-477c-b869-0722f6e25b1c None None] PATH_INFO: `/` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.510886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1234/2481] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 12:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:39:16.516699 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f45c5-17be-48d9-9613-50890aad2f2d None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.516993 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f45c5-17be-48d9-9613-50890aad2f2d None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.517266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f45c5-17be-48d9-9613-50890aad2f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.539266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-bd9f45c5-17be-48d9-9613-50890aad2f2d None None] MFA Rules not processed for user `e65026e01df242e7be11ed24214a4efa`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:16.540465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bd9f45c5-17be-48d9-9613-50890aad2f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:16.550425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1248/2482] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 12:39:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:16.564399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab767338-ac0e-477c-b869-0722f6e25b1c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:16.578101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab767338-ac0e-477c-b869-0722f6e25b1c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:16.583661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388dfb87-a708-4396-ba01-cf101cda002a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:16.584257 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dfb87-a708-4396-ba01-cf101cda002a service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.584535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dfb87-a708-4396-ba01-cf101cda002a service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.584837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dfb87-a708-4396-ba01-cf101cda002a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.598263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388dfb87-a708-4396-ba01-cf101cda002a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:16.599840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388dfb87-a708-4396-ba01-cf101cda002a service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:16.612352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1235/2483] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:16.619789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f45c5-17be-48d9-9613-50890aad2f2d None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:16.633149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f45c5-17be-48d9-9613-50890aad2f2d None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:16.640482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d81850-c46e-456a-9a5d-ee7f1108fde1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:16.641432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d81850-c46e-456a-9a5d-ee7f1108fde1 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.641777 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d81850-c46e-456a-9a5d-ee7f1108fde1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.642068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d81850-c46e-456a-9a5d-ee7f1108fde1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.645299 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-81d81850-c46e-456a-9a5d-ee7f1108fde1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:16.724690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d81850-c46e-456a-9a5d-ee7f1108fde1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:16.725286 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d81850-c46e-456a-9a5d-ee7f1108fde1 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:16.735795 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1249/2484] 10.222.0.27 () {60 vars in 1396 bytes} [Wed Aug 24 12:39:16 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:18.691713 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019df5e-c3a7-45a7-975b-686eb6dfe4bb None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:18.692191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019df5e-c3a7-45a7-975b-686eb6dfe4bb None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:18.692284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019df5e-c3a7-45a7-975b-686eb6dfe4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:18.715048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f019df5e-c3a7-45a7-975b-686eb6dfe4bb None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:18.716374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f019df5e-c3a7-45a7-975b-686eb6dfe4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:18.776961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1236/2485] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:39:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:18.789277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d81850-c46e-456a-9a5d-ee7f1108fde1 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:18.790720 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-81d81850-c46e-456a-9a5d-ee7f1108fde1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:18.805561 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:18.867148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d81850-c46e-456a-9a5d-ee7f1108fde1 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:18.872920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89999646-e6f8-4a90-9860-a7732cf21c5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:18.873510 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89999646-e6f8-4a90-9860-a7732cf21c5f service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:18.873783 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89999646-e6f8-4a90-9860-a7732cf21c5f service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:18.874049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89999646-e6f8-4a90-9860-a7732cf21c5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:18.882493 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89999646-e6f8-4a90-9860-a7732cf21c5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:18.884000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89999646-e6f8-4a90-9860-a7732cf21c5f service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:18.892244 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1250/2486] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:39:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:39:28.769871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=66 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:28.889358 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=26 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:45.791160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019df5e-c3a7-45a7-975b-686eb6dfe4bb None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:45.860604 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019df5e-c3a7-45a7-975b-686eb6dfe4bb None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:45.867278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:45.867854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:45.868134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:45.868401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] PATH_INFO: `/v3/users/889fbdaae5a440b3b0055146bc4cfd33` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:45.869718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=889fbdaae5a440b3b0055146bc4cfd33)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:45.872303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:45.930576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '889fbdaae5a440b3b0055146bc4cfd33'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.931344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '889fbdaae5a440b3b0055146bc4cfd33'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.931893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '889fbdaae5a440b3b0055146bc4cfd33'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.944777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '889fbdaae5a440b3b0055146bc4cfd33'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.956091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '889fbdaae5a440b3b0055146bc4cfd33'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.970171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1237/2487] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:45 2022] DELETE /identity/v3/users/889fbdaae5a440b3b0055146bc4cfd33 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:45.974723 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe58cc4-2d6d-44b0-8eb8-1d1de5ae1a26 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:45.975239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe58cc4-2d6d-44b0-8eb8-1d1de5ae1a26 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:45.975356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe58cc4-2d6d-44b0-8eb8-1d1de5ae1a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:45.998370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-cbe58cc4-2d6d-44b0-8eb8-1d1de5ae1a26 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:45.999632 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cbe58cc4-2d6d-44b0-8eb8-1d1de5ae1a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:46.061549 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1251/2488] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:46.071399 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.072903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:46.137433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da38713-ebcc-4eb0-a6fa-21803c64fb85 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.141870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239cb3db-39b8-454d-bae2-b811af8ca241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.142468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cb3db-39b8-454d-bae2-b811af8ca241 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.142741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cb3db-39b8-454d-bae2-b811af8ca241 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.143006 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cb3db-39b8-454d-bae2-b811af8ca241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.158351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239cb3db-39b8-454d-bae2-b811af8ca241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.159797 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239cb3db-39b8-454d-bae2-b811af8ca241 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.170777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1238/2489] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:46.476335 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe58cc4-2d6d-44b0-8eb8-1d1de5ae1a26 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.478137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cbe58cc4-2d6d-44b0-8eb8-1d1de5ae1a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:46.492916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe58cc4-2d6d-44b0-8eb8-1d1de5ae1a26 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.498609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.499258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.499519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.499798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2 admin admin] PATH_INFO: `/v3/projects/36393e8f816345a9b86d380bad8c0e55` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.501069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36393e8f816345a9b86d380bad8c0e55)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.502966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.553176 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36393e8f816345a9b86d380bad8c0e55', 'request_id': 'req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:46.562075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36393e8f816345a9b86d380bad8c0e55', 'request_id': 'req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:46.568545 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36393e8f816345a9b86d380bad8c0e55', 'request_id': 'req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:46.581337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2 admin admin] The token cache is being invalidate because project 36393e8f816345a9b86d380bad8c0e55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:46.581625 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:46.582336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-73e4045e-22bd-4bb1-9b58-a0d0b4849cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:46.583893 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1252/2490] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:46 2022] DELETE /identity/v3/projects/36393e8f816345a9b86d380bad8c0e55 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:46.588862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0d85b-50b4-4220-9a32-985aff8ef2f0 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.589152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0d85b-50b4-4220-9a32-985aff8ef2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.589477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0d85b-50b4-4220-9a32-985aff8ef2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.610984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-cca0d85b-50b4-4220-9a32-985aff8ef2f0 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:46.612203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cca0d85b-50b4-4220-9a32-985aff8ef2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:46.633764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:46.684515 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1239/2491] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:46.693832 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412844d2-7b72-4f14-b565-0cf98738f9cd None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.694116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412844d2-7b72-4f14-b565-0cf98738f9cd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.694390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412844d2-7b72-4f14-b565-0cf98738f9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.716226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-412844d2-7b72-4f14-b565-0cf98738f9cd None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:46.717491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-412844d2-7b72-4f14-b565-0cf98738f9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:46.730851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1253/2492] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:46.736380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0d85b-50b4-4220-9a32-985aff8ef2f0 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.746729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0d85b-50b4-4220-9a32-985aff8ef2f0 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.751888 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d21e93-48c3-4fcd-9576-8c2d030a57e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.752490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d21e93-48c3-4fcd-9576-8c2d030a57e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.752824 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d21e93-48c3-4fcd-9576-8c2d030a57e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.753102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d21e93-48c3-4fcd-9576-8c2d030a57e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.753779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d21e93-48c3-4fcd-9576-8c2d030a57e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:39:46.754079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d21e93-48c3-4fcd-9576-8c2d030a57e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.755768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d21e93-48c3-4fcd-9576-8c2d030a57e0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.768966 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1240/2493] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:39:46.773477 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412844d2-7b72-4f14-b565-0cf98738f9cd None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.783038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412844d2-7b72-4f14-b565-0cf98738f9cd None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.787818 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244b251-73c5-4650-a81b-484e31a5e8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.788407 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244b251-73c5-4650-a81b-484e31a5e8ac admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.788707 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244b251-73c5-4650-a81b-484e31a5e8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.788979 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244b251-73c5-4650-a81b-484e31a5e8ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.789593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244b251-73c5-4650-a81b-484e31a5e8ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.791504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244b251-73c5-4650-a81b-484e31a5e8ac admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.826423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1254/2494] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:46 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:46.827440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d21e93-48c3-4fcd-9576-8c2d030a57e0 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.837924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d21e93-48c3-4fcd-9576-8c2d030a57e0 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.899644 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f8219-77f1-44e6-b2ac-39c6b8334608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.900256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f8219-77f1-44e6-b2ac-39c6b8334608 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.900537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f8219-77f1-44e6-b2ac-39c6b8334608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.900845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f8219-77f1-44e6-b2ac-39c6b8334608 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.901474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f8219-77f1-44e6-b2ac-39c6b8334608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.903983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f8219-77f1-44e6-b2ac-39c6b8334608 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.931743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1241/2495] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:46 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:46.933083 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244b251-73c5-4650-a81b-484e31a5e8ac admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.943750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244b251-73c5-4650-a81b-484e31a5e8ac admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.948494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf94c30-9914-4688-938a-56c27985ba57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.949316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf94c30-9914-4688-938a-56c27985ba57 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.949613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf94c30-9914-4688-938a-56c27985ba57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.949872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf94c30-9914-4688-938a-56c27985ba57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.950601 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf94c30-9914-4688-938a-56c27985ba57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.952245 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf94c30-9914-4688-938a-56c27985ba57 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.965697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1255/2496] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:46.967010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f8219-77f1-44e6-b2ac-39c6b8334608 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.978666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f8219-77f1-44e6-b2ac-39c6b8334608 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.985670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c5f6a-a5c4-487a-9967-d4942e33c349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.986368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c5f6a-a5c4-487a-9967-d4942e33c349 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.986714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c5f6a-a5c4-487a-9967-d4942e33c349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.987091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c5f6a-a5c4-487a-9967-d4942e33c349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.988020 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c5f6a-a5c4-487a-9967-d4942e33c349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:47.001628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c5f6a-a5c4-487a-9967-d4942e33c349 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:47.015286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1242/2497] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:47.019711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf94c30-9914-4688-938a-56c27985ba57 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:47.033451 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf94c30-9914-4688-938a-56c27985ba57 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:47.038566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3a441-34d3-4f3a-beaf-7907bd9ad15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:47.039231 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3a441-34d3-4f3a-beaf-7907bd9ad15f admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:47.039508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3a441-34d3-4f3a-beaf-7907bd9ad15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:47.039775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3a441-34d3-4f3a-beaf-7907bd9ad15f admin admin] PATH_INFO: `/v3/projects/a489b947df404e25a523e24e1d9f4329/users/0e7af6df9d5a482b8230ca7e3c1f0fd3/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:47.055665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3a441-34d3-4f3a-beaf-7907bd9ad15f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a489b947df404e25a523e24e1d9f4329, user_id=0e7af6df9d5a482b8230ca7e3c1f0fd3, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:47.058366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3a441-34d3-4f3a-beaf-7907bd9ad15f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:47.073397 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1256/2498] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:47 2022] PUT /identity/v3/projects/a489b947df404e25a523e24e1d9f4329/users/0e7af6df9d5a482b8230ca7e3c1f0fd3/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:47.083730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7104ef9-5e97-463e-a0e1-94b04b0804c4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:47.084015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7104ef9-5e97-463e-a0e1-94b04b0804c4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:47.084300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7104ef9-5e97-463e-a0e1-94b04b0804c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:47.112278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c7104ef9-5e97-463e-a0e1-94b04b0804c4 None None] MFA Rules not processed for user `0e7af6df9d5a482b8230ca7e3c1f0fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:47.113635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c7104ef9-5e97-463e-a0e1-94b04b0804c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:47.183432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1243/2499] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:39:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:47.194954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3a441-34d3-4f3a-beaf-7907bd9ad15f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:47.197257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2ed3a441-34d3-4f3a-beaf-7907bd9ad15f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:47.225523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:47.274867 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3a441-34d3-4f3a-beaf-7907bd9ad15f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:47.276772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b6b85-47c5-4f4c-8e59-f7e8705aff01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:47.277404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b6b85-47c5-4f4c-8e59-f7e8705aff01 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:47.277681 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b6b85-47c5-4f4c-8e59-f7e8705aff01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:47.277951 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b6b85-47c5-4f4c-8e59-f7e8705aff01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:47.287021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b6b85-47c5-4f4c-8e59-f7e8705aff01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:47.288454 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b6b85-47c5-4f4c-8e59-f7e8705aff01 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:47.298887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1257/2500] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:57.180313 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:57.294224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=17 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:57.651379 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7104ef9-5e97-463e-a0e1-94b04b0804c4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:57.718766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7104ef9-5e97-463e-a0e1-94b04b0804c4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:57.723629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:57.724264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:57.724546 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:57.724844 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] PATH_INFO: `/v3/users/0e7af6df9d5a482b8230ca7e3c1f0fd3` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:57.725975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e7af6df9d5a482b8230ca7e3c1f0fd3)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:57.727912 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:57.780458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e7af6df9d5a482b8230ca7e3c1f0fd3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:57.781163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e7af6df9d5a482b8230ca7e3c1f0fd3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:57.781771 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e7af6df9d5a482b8230ca7e3c1f0fd3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:57.795484 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e7af6df9d5a482b8230ca7e3c1f0fd3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:57.806836 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e7af6df9d5a482b8230ca7e3c1f0fd3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:57.819744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1244/2501] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:57 2022] DELETE /identity/v3/users/0e7af6df9d5a482b8230ca7e3c1f0fd3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:57.825014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c0877-756d-4ecc-8291-ec49ca95d585 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:57.825292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c0877-756d-4ecc-8291-ec49ca95d585 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:57.825593 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c0877-756d-4ecc-8291-ec49ca95d585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:57.845812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8c1c0877-756d-4ecc-8291-ec49ca95d585 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:57.847038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8c1c0877-756d-4ecc-8291-ec49ca95d585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:57.911784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1258/2502] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:57.922608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:57.924215 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:57.988819 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f2be2-cc97-4d6c-9315-b74ba6372909 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:57.993037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1dbd9-baf9-410b-b07a-6b3e14ee23ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:57.993754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1dbd9-baf9-410b-b07a-6b3e14ee23ca service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:57.994040 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1dbd9-baf9-410b-b07a-6b3e14ee23ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:57.994310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1dbd9-baf9-410b-b07a-6b3e14ee23ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:58.004447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1dbd9-baf9-410b-b07a-6b3e14ee23ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:58.005920 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1dbd9-baf9-410b-b07a-6b3e14ee23ca service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:58.016641 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1245/2503] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:58.306322 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c0877-756d-4ecc-8291-ec49ca95d585 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:58.308027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8c1c0877-756d-4ecc-8291-ec49ca95d585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:58.321134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c0877-756d-4ecc-8291-ec49ca95d585 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:58.327488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:58.328128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:58.328406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:58.328674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] PATH_INFO: `/v3/projects/a489b947df404e25a523e24e1d9f4329` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:58.329923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a489b947df404e25a523e24e1d9f4329)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:58.331940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:58.385067 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a489b947df404e25a523e24e1d9f4329', 'request_id': 'req-f0463918-08e8-4b0a-b5d7-1edeffdbce12'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:58.394123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a489b947df404e25a523e24e1d9f4329', 'request_id': 'req-f0463918-08e8-4b0a-b5d7-1edeffdbce12'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:58.400839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a489b947df404e25a523e24e1d9f4329', 'request_id': 'req-f0463918-08e8-4b0a-b5d7-1edeffdbce12'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:58.413417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] The token cache is being invalidate because project a489b947df404e25a523e24e1d9f4329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:58.413812 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:58.414752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:58.425186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1259/2504] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:58 2022] DELETE /identity/v3/projects/a489b947df404e25a523e24e1d9f4329 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:58.528199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346febb-7260-4772-bcd4-2dacf0b64e3d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:58.528815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346febb-7260-4772-bcd4-2dacf0b64e3d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:58.528815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346febb-7260-4772-bcd4-2dacf0b64e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:58.550586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d346febb-7260-4772-bcd4-2dacf0b64e3d None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:58.551960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d346febb-7260-4772-bcd4-2dacf0b64e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:58.576995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:58.620452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1246/2505] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 24 12:39:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:58.633577 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:58.635239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:58.710668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0463918-08e8-4b0a-b5d7-1edeffdbce12 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:58.716445 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3368dd-7f08-45f6-a12d-0deda4ef2a15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:58.717260 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3368dd-7f08-45f6-a12d-0deda4ef2a15 service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:58.717580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3368dd-7f08-45f6-a12d-0deda4ef2a15 service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:58.717955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3368dd-7f08-45f6-a12d-0deda4ef2a15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:58.727644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3368dd-7f08-45f6-a12d-0deda4ef2a15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:58.729280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3368dd-7f08-45f6-a12d-0deda4ef2a15 service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:58.737836 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1260/2506] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:39:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:40:08.613581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=15 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:08.734130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:14.322571 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346febb-7260-4772-bcd4-2dacf0b64e3d None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:14.324548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d346febb-7260-4772-bcd4-2dacf0b64e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:14.390354 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346febb-7260-4772-bcd4-2dacf0b64e3d None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:14.394632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7e238-2f3b-4a3b-b587-bcf8159ecff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:14.395267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7e238-2f3b-4a3b-b587-bcf8159ecff4 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:14.395545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7e238-2f3b-4a3b-b587-bcf8159ecff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:14.395765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7e238-2f3b-4a3b-b587-bcf8159ecff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:14.397726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dea7e238-2f3b-4a3b-b587-bcf8159ecff4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:14.467580 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7e238-2f3b-4a3b-b587-bcf8159ecff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:14.469046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7e238-2f3b-4a3b-b587-bcf8159ecff4 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:14.480476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1247/2507] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:14.491240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3368dd-7f08-45f6-a12d-0deda4ef2a15 service swift] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:14.492679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fc3368dd-7f08-45f6-a12d-0deda4ef2a15 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:14.505310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3368dd-7f08-45f6-a12d-0deda4ef2a15 service swift] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:14.509508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa81bd-f01c-44cf-a0e8-c519fb9076f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:14.510307 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa81bd-f01c-44cf-a0e8-c519fb9076f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:14.510711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa81bd-f01c-44cf-a0e8-c519fb9076f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:14.511083 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa81bd-f01c-44cf-a0e8-c519fb9076f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:14.513146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9eaa81bd-f01c-44cf-a0e8-c519fb9076f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:14.526174 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa81bd-f01c-44cf-a0e8-c519fb9076f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:14.528067 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa81bd-f01c-44cf-a0e8-c519fb9076f7 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:14.538537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1261/2508] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:18.204688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7e238-2f3b-4a3b-b587-bcf8159ecff4 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:18.206432 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dea7e238-2f3b-4a3b-b587-bcf8159ecff4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:18.229513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7e238-2f3b-4a3b-b587-bcf8159ecff4 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:18.234750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:18.235496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:18.235951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:18.236182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] PATH_INFO: `/v3/users/85d8138d44bc40dd8f63ee4916ffbce9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:18.237422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d8138d44bc40dd8f63ee4916ffbce9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:18.239384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:18.318193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d8138d44bc40dd8f63ee4916ffbce9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.319286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d8138d44bc40dd8f63ee4916ffbce9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.320132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d8138d44bc40dd8f63ee4916ffbce9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.343230 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d8138d44bc40dd8f63ee4916ffbce9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.360062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d8138d44bc40dd8f63ee4916ffbce9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.385479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1248/2509] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:18 2022] DELETE /identity/v3/users/85d8138d44bc40dd8f63ee4916ffbce9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:18.452411 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa81bd-f01c-44cf-a0e8-c519fb9076f7 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:18.454260 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9eaa81bd-f01c-44cf-a0e8-c519fb9076f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:18.536088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:18.537697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:18.565442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa81bd-f01c-44cf-a0e8-c519fb9076f7 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:18.569584 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:18.634782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7de7eb-6941-4b7e-8780-e2f2cd4ce3d8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:18.641856 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ee708-3914-4cd6-b79b-1a512ed5aad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:18.642971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ee708-3914-4cd6-b79b-1a512ed5aad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:18.643460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ee708-3914-4cd6-b79b-1a512ed5aad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:18.643968 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ee708-3914-4cd6-b79b-1a512ed5aad5 admin admin] PATH_INFO: `/v3/users/b4a617fc13534a89ad50a5fdc7bf769a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:18.649851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ee708-3914-4cd6-b79b-1a512ed5aad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4a617fc13534a89ad50a5fdc7bf769a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:18.652878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ee708-3914-4cd6-b79b-1a512ed5aad5 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:18.654454 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd851b-7bc2-4155-8985-549f8d48b86c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:18.656103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd851b-7bc2-4155-8985-549f8d48b86c service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:18.656977 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd851b-7bc2-4155-8985-549f8d48b86c service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:18.657920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd851b-7bc2-4155-8985-549f8d48b86c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:18.661661 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3abd851b-7bc2-4155-8985-549f8d48b86c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:18.680466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd851b-7bc2-4155-8985-549f8d48b86c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:18.682385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd851b-7bc2-4155-8985-549f8d48b86c service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:18.694022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1262/2510] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:18.695333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd851b-7bc2-4155-8985-549f8d48b86c service placement] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:18.696810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3abd851b-7bc2-4155-8985-549f8d48b86c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:18.720407 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd851b-7bc2-4155-8985-549f8d48b86c service placement] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:18.723073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a62ee708-3914-4cd6-b79b-1a512ed5aad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4a617fc13534a89ad50a5fdc7bf769a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.723750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a62ee708-3914-4cd6-b79b-1a512ed5aad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4a617fc13534a89ad50a5fdc7bf769a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.724389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a62ee708-3914-4cd6-b79b-1a512ed5aad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4a617fc13534a89ad50a5fdc7bf769a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.725428 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:18.726249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:18.726726 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:18.727003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b admin admin] PATH_INFO: `/v3/projects/96fea11cfdd748bab1cdea46b073dc89` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:18.728149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b admin admin] RBAC: Authorizing `identity:delete_project(project_id=96fea11cfdd748bab1cdea46b073dc89)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:18.731587 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:18.738964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a62ee708-3914-4cd6-b79b-1a512ed5aad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4a617fc13534a89ad50a5fdc7bf769a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.750197 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a62ee708-3914-4cd6-b79b-1a512ed5aad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4a617fc13534a89ad50a5fdc7bf769a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.769210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1249/2511] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:18 2022] DELETE /identity/v3/users/b4a617fc13534a89ad50a5fdc7bf769a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:18.793401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96fea11cfdd748bab1cdea46b073dc89', 'request_id': 'req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.802683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96fea11cfdd748bab1cdea46b073dc89', 'request_id': 'req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.811264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96fea11cfdd748bab1cdea46b073dc89', 'request_id': 'req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.829800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b admin admin] The token cache is being invalidate because project 96fea11cfdd748bab1cdea46b073dc89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:18.829800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.829800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c0390f99-f1fd-41d4-a95e-eb5dfa15c90b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:18.832730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1263/2512] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:18 2022] DELETE /identity/v3/projects/96fea11cfdd748bab1cdea46b073dc89 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:18.837283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c930b-a2fa-44c1-b264-76d0690a2bfe None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:18.837660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c930b-a2fa-44c1-b264-76d0690a2bfe None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:18.838046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c930b-a2fa-44c1-b264-76d0690a2bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:18.860227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c73c930b-a2fa-44c1-b264-76d0690a2bfe None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:18.861507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c73c930b-a2fa-44c1-b264-76d0690a2bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:18.925023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1250/2513] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:18.935004 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0793fe-defe-42bf-889a-a2195dd03436 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:18.935298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0793fe-defe-42bf-889a-a2195dd03436 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:18.935571 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0793fe-defe-42bf-889a-a2195dd03436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:18.945922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:18.961523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6f0793fe-defe-42bf-889a-a2195dd03436 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:18.962773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6f0793fe-defe-42bf-889a-a2195dd03436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:18.967177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c930b-a2fa-44c1-b264-76d0690a2bfe None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:18.969608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c73c930b-a2fa-44c1-b264-76d0690a2bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:18.976875 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1264/2514] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:18.977916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0793fe-defe-42bf-889a-a2195dd03436 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:18.987392 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c930b-a2fa-44c1-b264-76d0690a2bfe None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:18.988042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0793fe-defe-42bf-889a-a2195dd03436 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:18.993614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f63927-dffa-4ded-baf9-03a55d589c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:18.994404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f63927-dffa-4ded-baf9-03a55d589c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:18.994585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f9d59-9ef8-4637-8e91-97a21973e01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:18.995303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f9d59-9ef8-4637-8e91-97a21973e01a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:18.995474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f63927-dffa-4ded-baf9-03a55d589c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:18.995742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f63927-dffa-4ded-baf9-03a55d589c81 admin admin] PATH_INFO: `/v3/projects/75590a43f50b48699fc0d5a892a99544` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:18.995851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f9d59-9ef8-4637-8e91-97a21973e01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:18.996116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f9d59-9ef8-4637-8e91-97a21973e01a admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:18.996815 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f9d59-9ef8-4637-8e91-97a21973e01a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:18.997577 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f63927-dffa-4ded-baf9-03a55d589c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75590a43f50b48699fc0d5a892a99544)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:18.997774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f9d59-9ef8-4637-8e91-97a21973e01a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:18.999283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f9d59-9ef8-4637-8e91-97a21973e01a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:18.999804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f63927-dffa-4ded-baf9-03a55d589c81 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.013078 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1265/2515] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:19.017604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f9d59-9ef8-4637-8e91-97a21973e01a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.029059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f9d59-9ef8-4637-8e91-97a21973e01a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.034492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714a777-abb5-4ece-b972-d569134ca60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.035091 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714a777-abb5-4ece-b972-d569134ca60f admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.035369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714a777-abb5-4ece-b972-d569134ca60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.035640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714a777-abb5-4ece-b972-d569134ca60f admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.036282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714a777-abb5-4ece-b972-d569134ca60f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.038268 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714a777-abb5-4ece-b972-d569134ca60f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.041663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:19.052700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f0f63927-dffa-4ded-baf9-03a55d589c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75590a43f50b48699fc0d5a892a99544', 'request_id': 'req-f0f63927-dffa-4ded-baf9-03a55d589c81'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:19.060902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f0f63927-dffa-4ded-baf9-03a55d589c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75590a43f50b48699fc0d5a892a99544', 'request_id': 'req-f0f63927-dffa-4ded-baf9-03a55d589c81'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:19.067510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f0f63927-dffa-4ded-baf9-03a55d589c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75590a43f50b48699fc0d5a892a99544', 'request_id': 'req-f0f63927-dffa-4ded-baf9-03a55d589c81'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:19.073613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1266/2516] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:19.074868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714a777-abb5-4ece-b972-d569134ca60f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.078464 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f0f63927-dffa-4ded-baf9-03a55d589c81 admin admin] The token cache is being invalidate because project 75590a43f50b48699fc0d5a892a99544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:19.078756 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f0f63927-dffa-4ded-baf9-03a55d589c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:19.079449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f0f63927-dffa-4ded-baf9-03a55d589c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:19.081192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1251/2517] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:18 2022] DELETE /identity/v3/projects/75590a43f50b48699fc0d5a892a99544 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:19.085893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d6523-b01b-49ac-be59-13c333b5aa53 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.086179 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d6523-b01b-49ac-be59-13c333b5aa53 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.086460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d6523-b01b-49ac-be59-13c333b5aa53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.107028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f61d6523-b01b-49ac-be59-13c333b5aa53 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:19.108302 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f61d6523-b01b-49ac-be59-13c333b5aa53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:19.137966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714a777-abb5-4ece-b972-d569134ca60f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.142844 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2045b-00b1-4d4c-b20a-02cb5b59944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.143489 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2045b-00b1-4d4c-b20a-02cb5b59944d admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.143755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2045b-00b1-4d4c-b20a-02cb5b59944d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.144032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2045b-00b1-4d4c-b20a-02cb5b59944d admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.144628 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2045b-00b1-4d4c-b20a-02cb5b59944d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.146627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2045b-00b1-4d4c-b20a-02cb5b59944d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.172409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1267/2518] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:19.172609 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1252/2519] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:19.173857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2045b-00b1-4d4c-b20a-02cb5b59944d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.175427 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-dae2045b-00b1-4d4c-b20a-02cb5b59944d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:19.178309 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94022-42e3-4908-b461-c10a65729159 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.178662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94022-42e3-4908-b461-c10a65729159 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.178940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94022-42e3-4908-b461-c10a65729159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.187727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2045b-00b1-4d4c-b20a-02cb5b59944d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.192264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67352269-dd82-4481-ba17-cbc431703d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.192877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67352269-dd82-4481-ba17-cbc431703d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.193151 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67352269-dd82-4481-ba17-cbc431703d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.193415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67352269-dd82-4481-ba17-cbc431703d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.194047 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67352269-dd82-4481-ba17-cbc431703d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.195659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67352269-dd82-4481-ba17-cbc431703d3d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.198381 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b1f94022-42e3-4908-b461-c10a65729159 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:19.199580 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b1f94022-42e3-4908-b461-c10a65729159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:19.206393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1268/2520] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:19.207718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1253/2521] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:19.210211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67352269-dd82-4481-ba17-cbc431703d3d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.213244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f94022-42e3-4908-b461-c10a65729159 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.220911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67352269-dd82-4481-ba17-cbc431703d3d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.224867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f94022-42e3-4908-b461-c10a65729159 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.225787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfbdf9-f054-40d8-88e4-5da663fc57f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.226391 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfbdf9-f054-40d8-88e4-5da663fc57f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.226669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfbdf9-f054-40d8-88e4-5da663fc57f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.226936 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfbdf9-f054-40d8-88e4-5da663fc57f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.227604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfbdf9-f054-40d8-88e4-5da663fc57f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.229356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfbdf9-f054-40d8-88e4-5da663fc57f3 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.229569 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a02e35-6eed-4b8c-800f-befb96002a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.230154 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a02e35-6eed-4b8c-800f-befb96002a8b admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.230430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a02e35-6eed-4b8c-800f-befb96002a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.230692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a02e35-6eed-4b8c-800f-befb96002a8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.231692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a02e35-6eed-4b8c-800f-befb96002a8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:19.231692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a02e35-6eed-4b8c-800f-befb96002a8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.233257 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a02e35-6eed-4b8c-800f-befb96002a8b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.243178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1269/2522] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:19.244404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfbdf9-f054-40d8-88e4-5da663fc57f3 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.246472 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1254/2523] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:40:19.250637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a02e35-6eed-4b8c-800f-befb96002a8b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.254414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfbdf9-f054-40d8-88e4-5da663fc57f3 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.259049 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10a8d4-e724-4e8a-99c3-5c5b116cbfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.259725 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10a8d4-e724-4e8a-99c3-5c5b116cbfbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.260033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10a8d4-e724-4e8a-99c3-5c5b116cbfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.260461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10a8d4-e724-4e8a-99c3-5c5b116cbfbc admin admin] PATH_INFO: `/v3/projects/a9000c16354a4405b498a5b2a228b905/users/105862091d7344118c1505d6f6b01f85/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.261546 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a02e35-6eed-4b8c-800f-befb96002a8b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.266784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e8b27-98d8-4716-9cc1-57b557f0882b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.267369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e8b27-98d8-4716-9cc1-57b557f0882b admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.267705 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e8b27-98d8-4716-9cc1-57b557f0882b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.267951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e8b27-98d8-4716-9cc1-57b557f0882b admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.268585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e8b27-98d8-4716-9cc1-57b557f0882b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.270522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e8b27-98d8-4716-9cc1-57b557f0882b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.275473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10a8d4-e724-4e8a-99c3-5c5b116cbfbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9000c16354a4405b498a5b2a228b905, user_id=105862091d7344118c1505d6f6b01f85, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.278255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10a8d4-e724-4e8a-99c3-5c5b116cbfbc admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.293193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1270/2524] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:19 2022] PUT /identity/v3/projects/a9000c16354a4405b498a5b2a228b905/users/105862091d7344118c1505d6f6b01f85/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:19.304262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd0e52-f7d1-42b3-bec9-5826daa88351 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.304590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd0e52-f7d1-42b3-bec9-5826daa88351 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.304949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd0e52-f7d1-42b3-bec9-5826daa88351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.310481 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1255/2525] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:19.311499 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e8b27-98d8-4716-9cc1-57b557f0882b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.334484 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6cfd0e52-f7d1-42b3-bec9-5826daa88351 None None] MFA Rules not processed for user `105862091d7344118c1505d6f6b01f85`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:19.335603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6cfd0e52-f7d1-42b3-bec9-5826daa88351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:19.378207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e8b27-98d8-4716-9cc1-57b557f0882b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.383054 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fee821-42db-4cca-8a27-c74575df6304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.383672 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fee821-42db-4cca-8a27-c74575df6304 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.383948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fee821-42db-4cca-8a27-c74575df6304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.384218 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fee821-42db-4cca-8a27-c74575df6304 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.384901 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fee821-42db-4cca-8a27-c74575df6304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.386964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fee821-42db-4cca-8a27-c74575df6304 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.390975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:19.403786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1271/2526] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:19.408013 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd0e52-f7d1-42b3-bec9-5826daa88351 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.410044 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1256/2527] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:19.414297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fee821-42db-4cca-8a27-c74575df6304 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.419532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd0e52-f7d1-42b3-bec9-5826daa88351 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.424926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ad4ab-657c-4138-869e-5e0ff377c93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.425444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fee821-42db-4cca-8a27-c74575df6304 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.425570 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ad4ab-657c-4138-869e-5e0ff377c93c admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.425851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ad4ab-657c-4138-869e-5e0ff377c93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.426108 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ad4ab-657c-4138-869e-5e0ff377c93c admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.426748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ad4ab-657c-4138-869e-5e0ff377c93c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.428755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ad4ab-657c-4138-869e-5e0ff377c93c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.430359 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ea20c-c477-4ccd-b865-b77b50434a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.430943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ea20c-c477-4ccd-b865-b77b50434a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.431233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ea20c-c477-4ccd-b865-b77b50434a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.431483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ea20c-c477-4ccd-b865-b77b50434a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.432233 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ea20c-c477-4ccd-b865-b77b50434a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.433857 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ea20c-c477-4ccd-b865-b77b50434a9a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.443964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1257/2528] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:19.448065 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ea20c-c477-4ccd-b865-b77b50434a9a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.453518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:19.459144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ea20c-c477-4ccd-b865-b77b50434a9a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.460687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1272/2529] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:19.464869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ad4ab-657c-4138-869e-5e0ff377c93c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.524208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c5138-7089-4b74-8248-168b17f0a1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.524833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c5138-7089-4b74-8248-168b17f0a1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.525106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c5138-7089-4b74-8248-168b17f0a1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.525370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c5138-7089-4b74-8248-168b17f0a1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.526101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c5138-7089-4b74-8248-168b17f0a1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.527847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c5138-7089-4b74-8248-168b17f0a1c1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.534796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ad4ab-657c-4138-869e-5e0ff377c93c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.537793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1258/2530] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:19.542274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c5138-7089-4b74-8248-168b17f0a1c1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.543620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a604c-499e-4e9d-96c0-91d439436a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.544236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a604c-499e-4e9d-96c0-91d439436a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.544513 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a604c-499e-4e9d-96c0-91d439436a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.544826 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a604c-499e-4e9d-96c0-91d439436a5e admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.545488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a604c-499e-4e9d-96c0-91d439436a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.547688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a604c-499e-4e9d-96c0-91d439436a5e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.551805 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c5138-7089-4b74-8248-168b17f0a1c1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.556742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1dacc-4139-4d75-923c-a76f1c7a1cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.557466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1dacc-4139-4d75-923c-a76f1c7a1cca admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.557741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1dacc-4139-4d75-923c-a76f1c7a1cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.558008 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1dacc-4139-4d75-923c-a76f1c7a1cca admin admin] PATH_INFO: `/v3/projects/5a432529d74947d18b99d5a771cd8f94/users/ae935f699bb1477e9c1138e9fff291e8/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.570235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1273/2531] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:19.574258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a604c-499e-4e9d-96c0-91d439436a5e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.576426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1dacc-4139-4d75-923c-a76f1c7a1cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a432529d74947d18b99d5a771cd8f94, user_id=ae935f699bb1477e9c1138e9fff291e8, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.579116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1dacc-4139-4d75-923c-a76f1c7a1cca admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.584097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a604c-499e-4e9d-96c0-91d439436a5e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.589909 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbb6c8-fa99-4798-88a9-a168e19cbbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.590450 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbb6c8-fa99-4798-88a9-a168e19cbbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.590726 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbb6c8-fa99-4798-88a9-a168e19cbbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.591080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbb6c8-fa99-4798-88a9-a168e19cbbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.591873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbb6c8-fa99-4798-88a9-a168e19cbbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.593604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbb6c8-fa99-4798-88a9-a168e19cbbd2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.596259 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1259/2532] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:19 2022] PUT /identity/v3/projects/5a432529d74947d18b99d5a771cd8f94/users/ae935f699bb1477e9c1138e9fff291e8/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:19.604860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1274/2533] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:19.606773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cab188-6899-4f05-b64d-5402907554f8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.607143 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cab188-6899-4f05-b64d-5402907554f8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.607483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cab188-6899-4f05-b64d-5402907554f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.608788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbb6c8-fa99-4798-88a9-a168e19cbbd2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.619074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbb6c8-fa99-4798-88a9-a168e19cbbd2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.637902 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-51cab188-6899-4f05-b64d-5402907554f8 None None] MFA Rules not processed for user `ae935f699bb1477e9c1138e9fff291e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:19.639344 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-51cab188-6899-4f05-b64d-5402907554f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:19.681775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddb93f-9b9c-4844-9945-3e6a0aa2df8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.682487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddb93f-9b9c-4844-9945-3e6a0aa2df8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.682750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddb93f-9b9c-4844-9945-3e6a0aa2df8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.683016 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddb93f-9b9c-4844-9945-3e6a0aa2df8f admin admin] PATH_INFO: `/v3/projects/f2fdef372eb34b47a07facf95eaa4768/users/fd56de52c58f4f369a426d94d759e294/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.697607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbddb93f-9b9c-4844-9945-3e6a0aa2df8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2fdef372eb34b47a07facf95eaa4768, user_id=fd56de52c58f4f369a426d94d759e294, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.700459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbddb93f-9b9c-4844-9945-3e6a0aa2df8f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.707980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1260/2534] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:19.712529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cab188-6899-4f05-b64d-5402907554f8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.717417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1275/2535] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:19 2022] PUT /identity/v3/projects/f2fdef372eb34b47a07facf95eaa4768/users/fd56de52c58f4f369a426d94d759e294/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:19.721184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddb93f-9b9c-4844-9945-3e6a0aa2df8f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.723392 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cab188-6899-4f05-b64d-5402907554f8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.754424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:19.780016 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e38f2-7ce7-40e4-93b4-3bf7a78545e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.780625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e38f2-7ce7-40e4-93b4-3bf7a78545e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.780924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e38f2-7ce7-40e4-93b4-3bf7a78545e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.781192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e38f2-7ce7-40e4-93b4-3bf7a78545e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.781813 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e38f2-7ce7-40e4-93b4-3bf7a78545e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.783717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e38f2-7ce7-40e4-93b4-3bf7a78545e9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.785604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddb93f-9b9c-4844-9945-3e6a0aa2df8f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.791439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842f251-1456-4cd2-94c4-cf77204af22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.792594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842f251-1456-4cd2-94c4-cf77204af22a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.792904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842f251-1456-4cd2-94c4-cf77204af22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.793337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842f251-1456-4cd2-94c4-cf77204af22a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.794292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842f251-1456-4cd2-94c4-cf77204af22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.794423 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:19.795749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842f251-1456-4cd2-94c4-cf77204af22a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.807240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1276/2536] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:19.811396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842f251-1456-4cd2-94c4-cf77204af22a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.816809 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1261/2537] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:19.818161 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e38f2-7ce7-40e4-93b4-3bf7a78545e9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.822306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842f251-1456-4cd2-94c4-cf77204af22a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.881535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e38f2-7ce7-40e4-93b4-3bf7a78545e9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.882297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d4d4c-da29-457e-8984-0c3d2ffa231f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.882971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d4d4c-da29-457e-8984-0c3d2ffa231f admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.883256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d4d4c-da29-457e-8984-0c3d2ffa231f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.883530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d4d4c-da29-457e-8984-0c3d2ffa231f admin admin] PATH_INFO: `/v3/projects/f2fdef372eb34b47a07facf95eaa4768/users/fd56de52c58f4f369a426d94d759e294/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.885624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d4d4c-da29-457e-8984-0c3d2ffa231f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2fdef372eb34b47a07facf95eaa4768, user_id=fd56de52c58f4f369a426d94d759e294, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.887382 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2305a31-c76b-44ba-97a7-6a53890d3fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:19.888021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2305a31-c76b-44ba-97a7-6a53890d3fcc admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.888289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d4d4c-da29-457e-8984-0c3d2ffa231f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.888413 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2305a31-c76b-44ba-97a7-6a53890d3fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.888553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2305a31-c76b-44ba-97a7-6a53890d3fcc admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.889249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2305a31-c76b-44ba-97a7-6a53890d3fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:19.891256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2305a31-c76b-44ba-97a7-6a53890d3fcc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:19.899178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1277/2538] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:19 2022] PUT /identity/v3/projects/f2fdef372eb34b47a07facf95eaa4768/users/fd56de52c58f4f369a426d94d759e294/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:19.911360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec583d4-6f6e-469c-875b-a9cbe9c8848b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:19.911681 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec583d4-6f6e-469c-875b-a9cbe9c8848b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:19.911946 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec583d4-6f6e-469c-875b-a9cbe9c8848b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:19.916321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1262/2539] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:19.922362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2305a31-c76b-44ba-97a7-6a53890d3fcc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:19.932805 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2305a31-c76b-44ba-97a7-6a53890d3fcc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:19.941579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5ec583d4-6f6e-469c-875b-a9cbe9c8848b None None] MFA Rules not processed for user `fd56de52c58f4f369a426d94d759e294`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:19.942792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5ec583d4-6f6e-469c-875b-a9cbe9c8848b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:20.001463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e18d5-17a9-4259-9896-d4aeb4f231da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:20.002228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e18d5-17a9-4259-9896-d4aeb4f231da admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:20.002496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e18d5-17a9-4259-9896-d4aeb4f231da admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:20.002828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e18d5-17a9-4259-9896-d4aeb4f231da admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:20.003506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e18d5-17a9-4259-9896-d4aeb4f231da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:20.005173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e18d5-17a9-4259-9896-d4aeb4f231da admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:20.016542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1263/2540] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:20.020783 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e18d5-17a9-4259-9896-d4aeb4f231da admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:20.026221 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1278/2541] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:20.031893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e18d5-17a9-4259-9896-d4aeb4f231da admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:20.036638 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd2d87-c882-4cbc-b9ef-74dbe2a48c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:20.037338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd2d87-c882-4cbc-b9ef-74dbe2a48c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:20.037599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec583d4-6f6e-469c-875b-a9cbe9c8848b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:20.037709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd2d87-c882-4cbc-b9ef-74dbe2a48c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:20.037872 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd2d87-c882-4cbc-b9ef-74dbe2a48c88 admin admin] PATH_INFO: `/v3/projects/4c2855631c6f4f34aad1da3045d63ff8/users/21d72a210047444192a89617a90086e5/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:20.038974 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5ec583d4-6f6e-469c-875b-a9cbe9c8848b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:20.053833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd2d87-c882-4cbc-b9ef-74dbe2a48c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c2855631c6f4f34aad1da3045d63ff8, user_id=21d72a210047444192a89617a90086e5, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:20.056617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd2d87-c882-4cbc-b9ef-74dbe2a48c88 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:20.070646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1264/2542] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:20 2022] PUT /identity/v3/projects/4c2855631c6f4f34aad1da3045d63ff8/users/21d72a210047444192a89617a90086e5/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:20.074949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd2d87-c882-4cbc-b9ef-74dbe2a48c88 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:20.097766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:20.101902 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec583d4-6f6e-469c-875b-a9cbe9c8848b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:20.127172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:20.140618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd2d87-c882-4cbc-b9ef-74dbe2a48c88 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:20.145692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d84f6-b97e-4dac-8a71-394545e94e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:20.146310 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d84f6-b97e-4dac-8a71-394545e94e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:20.146586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d84f6-b97e-4dac-8a71-394545e94e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:20.146980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d84f6-b97e-4dac-8a71-394545e94e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:20.147635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d84f6-b97e-4dac-8a71-394545e94e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:20.149419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d84f6-b97e-4dac-8a71-394545e94e29 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:20.153695 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c66e3-b47f-4a54-8e9c-fc73924de3ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:20.154343 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c66e3-b47f-4a54-8e9c-fc73924de3ef service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:20.154617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c66e3-b47f-4a54-8e9c-fc73924de3ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:20.154898 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c66e3-b47f-4a54-8e9c-fc73924de3ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:20.160557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1265/2543] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:20.164612 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d84f6-b97e-4dac-8a71-394545e94e29 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:20.175220 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d84f6-b97e-4dac-8a71-394545e94e29 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:20.180631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456c50f-0710-4a95-bfd6-bd068f92ec13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:20.181297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456c50f-0710-4a95-bfd6-bd068f92ec13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:20.181608 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456c50f-0710-4a95-bfd6-bd068f92ec13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:20.181932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456c50f-0710-4a95-bfd6-bd068f92ec13 admin admin] PATH_INFO: `/v3/projects/4c2855631c6f4f34aad1da3045d63ff8/users/21d72a210047444192a89617a90086e5/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:20.184125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456c50f-0710-4a95-bfd6-bd068f92ec13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c2855631c6f4f34aad1da3045d63ff8, user_id=21d72a210047444192a89617a90086e5, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:20.186815 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456c50f-0710-4a95-bfd6-bd068f92ec13 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:20.198102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1266/2544] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:20 2022] PUT /identity/v3/projects/4c2855631c6f4f34aad1da3045d63ff8/users/21d72a210047444192a89617a90086e5/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:20.210206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013219f2-8b67-4746-97a6-ab50a21c050b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:20.210909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013219f2-8b67-4746-97a6-ab50a21c050b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:20.210909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013219f2-8b67-4746-97a6-ab50a21c050b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:20.220822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c66e3-b47f-4a54-8e9c-fc73924de3ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:20.222394 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c66e3-b47f-4a54-8e9c-fc73924de3ef service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:20.233100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1279/2545] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:40:20.242081 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-013219f2-8b67-4746-97a6-ab50a21c050b None None] MFA Rules not processed for user `21d72a210047444192a89617a90086e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:20.243438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-013219f2-8b67-4746-97a6-ab50a21c050b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:20.319559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c66e3-b47f-4a54-8e9c-fc73924de3ef service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:20.324004 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1267/2546] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:20.333411 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c66e3-b47f-4a54-8e9c-fc73924de3ef service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:20.335764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013219f2-8b67-4746-97a6-ab50a21c050b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:20.337152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-013219f2-8b67-4746-97a6-ab50a21c050b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:20.384141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1188c-ec68-4e17-ab95-3bf0824ee2ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:20.384837 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1188c-ec68-4e17-ab95-3bf0824ee2ad service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:20.385228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1188c-ec68-4e17-ab95-3bf0824ee2ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:20.385441 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1188c-ec68-4e17-ab95-3bf0824ee2ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:20.392834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013219f2-8b67-4746-97a6-ab50a21c050b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:20.396376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf52ec8-5271-4d9a-abdd-1f55795bec91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:20.397049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf52ec8-5271-4d9a-abdd-1f55795bec91 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:20.397323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf52ec8-5271-4d9a-abdd-1f55795bec91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:20.397587 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf52ec8-5271-4d9a-abdd-1f55795bec91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:20.407237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf52ec8-5271-4d9a-abdd-1f55795bec91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:20.408834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf52ec8-5271-4d9a-abdd-1f55795bec91 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:20.419039 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1268/2547] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:20 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:40:20.447853 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1188c-ec68-4e17-ab95-3bf0824ee2ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:20.449674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1188c-ec68-4e17-ab95-3bf0824ee2ad service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:20.459019 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1280/2548] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:40:20.905456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf52ec8-5271-4d9a-abdd-1f55795bec91 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:20.975258 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf52ec8-5271-4d9a-abdd-1f55795bec91 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:20.980587 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:20.981416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:20.981521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:20.981793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] PATH_INFO: `/v3/users/ae935f699bb1477e9c1138e9fff291e8` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:20.982962 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae935f699bb1477e9c1138e9fff291e8)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:20.984972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:20.987697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:21.036184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae935f699bb1477e9c1138e9fff291e8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.036896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae935f699bb1477e9c1138e9fff291e8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.037548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae935f699bb1477e9c1138e9fff291e8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.050322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae935f699bb1477e9c1138e9fff291e8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.060486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae935f699bb1477e9c1138e9fff291e8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.073158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1269/2549] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:20 2022] DELETE /identity/v3/users/ae935f699bb1477e9c1138e9fff291e8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:21.077902 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d94f5-bdff-4a8a-a351-f0a21ef76e81 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:21.078410 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d94f5-bdff-4a8a-a351-f0a21ef76e81 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:21.078521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d94f5-bdff-4a8a-a351-f0a21ef76e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:21.098020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-130d94f5-bdff-4a8a-a351-f0a21ef76e81 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:21.099275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-130d94f5-bdff-4a8a-a351-f0a21ef76e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:21.103573 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:21.163317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1281/2550] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:21.174514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:21.176069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:21.236977 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a689a10-4e45-4e51-9108-8b4ce1440d4d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:21.241568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7d2d-02b9-4252-adfc-c3a119cc5514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:21.242199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7d2d-02b9-4252-adfc-c3a119cc5514 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:21.242472 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7d2d-02b9-4252-adfc-c3a119cc5514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:21.242739 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7d2d-02b9-4252-adfc-c3a119cc5514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:21.253431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a7d2d-02b9-4252-adfc-c3a119cc5514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:21.254883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a7d2d-02b9-4252-adfc-c3a119cc5514 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:21.264828 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1270/2551] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:21.566351 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d94f5-bdff-4a8a-a351-f0a21ef76e81 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:21.569065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-130d94f5-bdff-4a8a-a351-f0a21ef76e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:21.585532 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d94f5-bdff-4a8a-a351-f0a21ef76e81 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:21.591071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:21.592519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:21.592850 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:21.593331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] PATH_INFO: `/v3/projects/5a432529d74947d18b99d5a771cd8f94` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:21.594839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a432529d74947d18b99d5a771cd8f94)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:21.596938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:21.648518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a432529d74947d18b99d5a771cd8f94', 'request_id': 'req-fd1211a4-0285-4ae9-945e-6cc29c7ca439'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.658835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a432529d74947d18b99d5a771cd8f94', 'request_id': 'req-fd1211a4-0285-4ae9-945e-6cc29c7ca439'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.666125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a432529d74947d18b99d5a771cd8f94', 'request_id': 'req-fd1211a4-0285-4ae9-945e-6cc29c7ca439'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.677582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] The token cache is being invalidate because project 5a432529d74947d18b99d5a771cd8f94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:21.677887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.678589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.680172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1282/2552] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:21 2022] DELETE /identity/v3/projects/5a432529d74947d18b99d5a771cd8f94 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:21.684636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7d2d-02b9-4252-adfc-c3a119cc5514 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:21.694405 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7d2d-02b9-4252-adfc-c3a119cc5514 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:21.755394 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899d49b-7d2c-45b3-8162-697e29862009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:21.755956 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899d49b-7d2c-45b3-8162-697e29862009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:21.756234 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899d49b-7d2c-45b3-8162-697e29862009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:21.756507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899d49b-7d2c-45b3-8162-697e29862009 admin admin] PATH_INFO: `/v3/users/21d72a210047444192a89617a90086e5` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:21.757715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899d49b-7d2c-45b3-8162-697e29862009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21d72a210047444192a89617a90086e5)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:21.759659 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899d49b-7d2c-45b3-8162-697e29862009 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:21.790074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:21.814632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b899d49b-7d2c-45b3-8162-697e29862009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21d72a210047444192a89617a90086e5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.815299 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b899d49b-7d2c-45b3-8162-697e29862009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21d72a210047444192a89617a90086e5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.815879 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b899d49b-7d2c-45b3-8162-697e29862009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21d72a210047444192a89617a90086e5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.828950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b899d49b-7d2c-45b3-8162-697e29862009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21d72a210047444192a89617a90086e5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.847951 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-b899d49b-7d2c-45b3-8162-697e29862009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21d72a210047444192a89617a90086e5'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:21.861806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1271/2553] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:21 2022] DELETE /identity/v3/users/21d72a210047444192a89617a90086e5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:22.162787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:22.164339 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:22.179939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1211a4-0285-4ae9-945e-6cc29c7ca439 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:22.244467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db441ad-4d6d-4331-957c-389e0ec8e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:22.245191 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db441ad-4d6d-4331-957c-389e0ec8e316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:22.245609 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db441ad-4d6d-4331-957c-389e0ec8e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:22.245901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db441ad-4d6d-4331-957c-389e0ec8e316 admin admin] PATH_INFO: `/v3/projects/4c2855631c6f4f34aad1da3045d63ff8` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:22.247697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db441ad-4d6d-4331-957c-389e0ec8e316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c2855631c6f4f34aad1da3045d63ff8)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:22.250845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db441ad-4d6d-4331-957c-389e0ec8e316 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:22.298033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:22.310061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6db441ad-4d6d-4331-957c-389e0ec8e316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c2855631c6f4f34aad1da3045d63ff8', 'request_id': 'req-6db441ad-4d6d-4331-957c-389e0ec8e316'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:22.320486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6db441ad-4d6d-4331-957c-389e0ec8e316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c2855631c6f4f34aad1da3045d63ff8', 'request_id': 'req-6db441ad-4d6d-4331-957c-389e0ec8e316'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:22.328450 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6db441ad-4d6d-4331-957c-389e0ec8e316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c2855631c6f4f34aad1da3045d63ff8', 'request_id': 'req-6db441ad-4d6d-4331-957c-389e0ec8e316'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:22.342103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6db441ad-4d6d-4331-957c-389e0ec8e316 admin admin] The token cache is being invalidate because project 4c2855631c6f4f34aad1da3045d63ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:22.342547 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6db441ad-4d6d-4331-957c-389e0ec8e316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:22.343332 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-6db441ad-4d6d-4331-957c-389e0ec8e316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:22.345023 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1283/2554] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:22 2022] DELETE /identity/v3/projects/4c2855631c6f4f34aad1da3045d63ff8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:22.351207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec95049-bab4-4517-bfde-e2cb4ca40662 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:22.351703 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec95049-bab4-4517-bfde-e2cb4ca40662 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:22.351703 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec95049-bab4-4517-bfde-e2cb4ca40662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:22.375158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-dec95049-bab4-4517-bfde-e2cb4ca40662 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:22.376368 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-dec95049-bab4-4517-bfde-e2cb4ca40662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:22.455439 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1272/2555] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:22.463558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c06794-c9d5-42fd-91ea-307065ad4fe9 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:22.463852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c06794-c9d5-42fd-91ea-307065ad4fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:22.464126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c06794-c9d5-42fd-91ea-307065ad4fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:22.484813 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c6c06794-c9d5-42fd-91ea-307065ad4fe9 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:22.486028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c6c06794-c9d5-42fd-91ea-307065ad4fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:22.493943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1284/2556] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:22.498614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec95049-bab4-4517-bfde-e2cb4ca40662 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:22.508474 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec95049-bab4-4517-bfde-e2cb4ca40662 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:22.513834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef9347-61f0-40ad-9d58-fb73ebd333fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:22.514419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef9347-61f0-40ad-9d58-fb73ebd333fe admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:22.514674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef9347-61f0-40ad-9d58-fb73ebd333fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:22.514935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef9347-61f0-40ad-9d58-fb73ebd333fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:22.515568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef9347-61f0-40ad-9d58-fb73ebd333fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:22.515837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef9347-61f0-40ad-9d58-fb73ebd333fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:22.517517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef9347-61f0-40ad-9d58-fb73ebd333fe admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:22.529563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1273/2557] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:22.533841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c06794-c9d5-42fd-91ea-307065ad4fe9 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:22.544072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c06794-c9d5-42fd-91ea-307065ad4fe9 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:22.549410 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee8b7e-bf63-47f3-98c8-0fec353b402a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:22.549984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee8b7e-bf63-47f3-98c8-0fec353b402a admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:22.550256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee8b7e-bf63-47f3-98c8-0fec353b402a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:22.550518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee8b7e-bf63-47f3-98c8-0fec353b402a admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:22.551165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee8b7e-bf63-47f3-98c8-0fec353b402a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:22.553119 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee8b7e-bf63-47f3-98c8-0fec353b402a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:22.588957 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1285/2558] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:22 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:22.593704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef9347-61f0-40ad-9d58-fb73ebd333fe admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:22.607514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef9347-61f0-40ad-9d58-fb73ebd333fe admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:22.671747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1932501-7700-43de-a487-ff96bd8ef6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:22.672498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1932501-7700-43de-a487-ff96bd8ef6cf admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:22.672988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1932501-7700-43de-a487-ff96bd8ef6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:22.673259 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1932501-7700-43de-a487-ff96bd8ef6cf admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:22.673871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1932501-7700-43de-a487-ff96bd8ef6cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:22.675877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1932501-7700-43de-a487-ff96bd8ef6cf admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:22.701120 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1274/2559] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:22 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:22.706237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee8b7e-bf63-47f3-98c8-0fec353b402a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:22.718729 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee8b7e-bf63-47f3-98c8-0fec353b402a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:22.725139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274615af-dff1-4f63-b4c7-c7a0d213c4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:22.725738 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274615af-dff1-4f63-b4c7-c7a0d213c4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:22.726011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274615af-dff1-4f63-b4c7-c7a0d213c4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:22.726275 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274615af-dff1-4f63-b4c7-c7a0d213c4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:22.726916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274615af-dff1-4f63-b4c7-c7a0d213c4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:22.728550 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274615af-dff1-4f63-b4c7-c7a0d213c4a9 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:22.739048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1286/2560] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:22.743169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1932501-7700-43de-a487-ff96bd8ef6cf admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:22.754156 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1932501-7700-43de-a487-ff96bd8ef6cf admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:22.760704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788d449-4897-4683-b301-d3407ded0e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:22.761290 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788d449-4897-4683-b301-d3407ded0e5f admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:22.761547 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788d449-4897-4683-b301-d3407ded0e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:22.761811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788d449-4897-4683-b301-d3407ded0e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:22.762450 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788d449-4897-4683-b301-d3407ded0e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:22.764049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788d449-4897-4683-b301-d3407ded0e5f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:22.774791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1275/2561] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:22.780028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274615af-dff1-4f63-b4c7-c7a0d213c4a9 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:22.797737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274615af-dff1-4f63-b4c7-c7a0d213c4a9 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:22.806645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74624b87-9656-4935-b3f9-385a0b1022d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:22.807668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74624b87-9656-4935-b3f9-385a0b1022d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:22.807946 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74624b87-9656-4935-b3f9-385a0b1022d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:22.808217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74624b87-9656-4935-b3f9-385a0b1022d6 admin admin] PATH_INFO: `/v3/projects/9b3ecd2d2d064faabb5807733e72b917/users/821b1410a2054952812b89792c38d937/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:22.824872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74624b87-9656-4935-b3f9-385a0b1022d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b3ecd2d2d064faabb5807733e72b917, user_id=821b1410a2054952812b89792c38d937, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:22.827548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74624b87-9656-4935-b3f9-385a0b1022d6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:22.847691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1287/2562] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:22 2022] PUT /identity/v3/projects/9b3ecd2d2d064faabb5807733e72b917/users/821b1410a2054952812b89792c38d937/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:22.861711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c23ad-92a1-44ac-a2ff-275b0083a177 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:22.861996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c23ad-92a1-44ac-a2ff-275b0083a177 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:22.862266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c23ad-92a1-44ac-a2ff-275b0083a177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:22.882926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:22.894437 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d53c23ad-92a1-44ac-a2ff-275b0083a177 None None] MFA Rules not processed for user `821b1410a2054952812b89792c38d937`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:22.896718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d53c23ad-92a1-44ac-a2ff-275b0083a177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:22.964692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1276/2563] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:22.969556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74624b87-9656-4935-b3f9-385a0b1022d6 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:23.035861 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74624b87-9656-4935-b3f9-385a0b1022d6 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:23.042784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65332d7a-6abb-4329-99d1-8559f4ee1cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:23.043427 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65332d7a-6abb-4329-99d1-8559f4ee1cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:23.043711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65332d7a-6abb-4329-99d1-8559f4ee1cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:23.043973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65332d7a-6abb-4329-99d1-8559f4ee1cd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:23.044600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65332d7a-6abb-4329-99d1-8559f4ee1cd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:23.046989 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65332d7a-6abb-4329-99d1-8559f4ee1cd8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:23.078157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1288/2564] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:22 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:23.082509 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c23ad-92a1-44ac-a2ff-275b0083a177 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:23.094906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c23ad-92a1-44ac-a2ff-275b0083a177 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:23.159512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5148d-cbef-4e5c-a3cf-bc04bee65ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:23.160205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5148d-cbef-4e5c-a3cf-bc04bee65ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:23.160468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5148d-cbef-4e5c-a3cf-bc04bee65ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:23.160776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5148d-cbef-4e5c-a3cf-bc04bee65ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:23.161445 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5148d-cbef-4e5c-a3cf-bc04bee65ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:23.163469 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5148d-cbef-4e5c-a3cf-bc04bee65ff0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:23.191079 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1277/2565] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:23 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:23.197409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65332d7a-6abb-4329-99d1-8559f4ee1cd8 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:23.208457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65332d7a-6abb-4329-99d1-8559f4ee1cd8 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:23.213497 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2efe3b-89db-418c-9a98-7b31838da5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:23.214225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2efe3b-89db-418c-9a98-7b31838da5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:23.214565 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2efe3b-89db-418c-9a98-7b31838da5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:23.214919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2efe3b-89db-418c-9a98-7b31838da5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:23.215631 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2efe3b-89db-418c-9a98-7b31838da5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:23.217395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2efe3b-89db-418c-9a98-7b31838da5b7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:23.229095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1289/2566] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:23.233744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5148d-cbef-4e5c-a3cf-bc04bee65ff0 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:23.244639 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5148d-cbef-4e5c-a3cf-bc04bee65ff0 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:23.252841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6c6a1-38d5-47ad-8b49-a1fe1914be58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:23.253511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6c6a1-38d5-47ad-8b49-a1fe1914be58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:23.253784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6c6a1-38d5-47ad-8b49-a1fe1914be58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:23.254051 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6c6a1-38d5-47ad-8b49-a1fe1914be58 admin admin] PATH_INFO: `/v3/projects/17ff3395e0bd4380aca56f9b3ad4fd3c/users/7d960d714755477cbf5351531c7d5e4a/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:23.270791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6c6a1-38d5-47ad-8b49-a1fe1914be58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ff3395e0bd4380aca56f9b3ad4fd3c, user_id=7d960d714755477cbf5351531c7d5e4a, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:23.273570 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6c6a1-38d5-47ad-8b49-a1fe1914be58 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:23.288777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1278/2567] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:23 2022] PUT /identity/v3/projects/17ff3395e0bd4380aca56f9b3ad4fd3c/users/7d960d714755477cbf5351531c7d5e4a/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:23.294597 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2efe3b-89db-418c-9a98-7b31838da5b7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:23.306082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2efe3b-89db-418c-9a98-7b31838da5b7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:23.344830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:23.376667 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830b340-c14b-404b-b367-41221a6abe14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:23.377298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830b340-c14b-404b-b367-41221a6abe14 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:23.377570 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830b340-c14b-404b-b367-41221a6abe14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:23.377839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830b340-c14b-404b-b367-41221a6abe14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:23.378541 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830b340-c14b-404b-b367-41221a6abe14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:23.380153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830b340-c14b-404b-b367-41221a6abe14 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:23.390331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1290/2568] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:23.396802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6c6a1-38d5-47ad-8b49-a1fe1914be58 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:23.409632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6c6a1-38d5-47ad-8b49-a1fe1914be58 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:23.414635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa00f2a-ed3f-4577-81d7-c0758ef96f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:23.415282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa00f2a-ed3f-4577-81d7-c0758ef96f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:23.415884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa00f2a-ed3f-4577-81d7-c0758ef96f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:23.416216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa00f2a-ed3f-4577-81d7-c0758ef96f26 admin admin] PATH_INFO: `/v3/projects/17ff3395e0bd4380aca56f9b3ad4fd3c/users/7d960d714755477cbf5351531c7d5e4a/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:23.418496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa00f2a-ed3f-4577-81d7-c0758ef96f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ff3395e0bd4380aca56f9b3ad4fd3c, user_id=7d960d714755477cbf5351531c7d5e4a, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:23.421176 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa00f2a-ed3f-4577-81d7-c0758ef96f26 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:23.432026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1279/2569] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:23 2022] PUT /identity/v3/projects/17ff3395e0bd4380aca56f9b3ad4fd3c/users/7d960d714755477cbf5351531c7d5e4a/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:23.443407 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9af937-e2d6-4681-abcc-42d4e1b0886b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:23.443760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9af937-e2d6-4681-abcc-42d4e1b0886b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:23.444156 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9af937-e2d6-4681-abcc-42d4e1b0886b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:23.481491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3b9af937-e2d6-4681-abcc-42d4e1b0886b None None] MFA Rules not processed for user `7d960d714755477cbf5351531c7d5e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:23.482792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3b9af937-e2d6-4681-abcc-42d4e1b0886b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:23.555100 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1291/2570] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:23.567204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa00f2a-ed3f-4577-81d7-c0758ef96f26 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:23.569205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5fa00f2a-ed3f-4577-81d7-c0758ef96f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:23.637552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa00f2a-ed3f-4577-81d7-c0758ef96f26 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:23.641646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49fccc-26f3-46f2-ad96-7104af771ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:23.642248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fccc-26f3-46f2-ad96-7104af771ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:23.642545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fccc-26f3-46f2-ad96-7104af771ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:23.642779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49fccc-26f3-46f2-ad96-7104af771ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:23.648749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:23.653798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49fccc-26f3-46f2-ad96-7104af771ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:23.655429 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49fccc-26f3-46f2-ad96-7104af771ac5 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:23.666140 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1280/2571] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:23.753968 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9af937-e2d6-4681-abcc-42d4e1b0886b None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:23.765416 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9af937-e2d6-4681-abcc-42d4e1b0886b None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:23.769924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a3ff4-98bc-44a8-85b5-9300bf7d4acb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:23.770525 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a3ff4-98bc-44a8-85b5-9300bf7d4acb service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:23.770803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a3ff4-98bc-44a8-85b5-9300bf7d4acb service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:23.771077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a3ff4-98bc-44a8-85b5-9300bf7d4acb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:23.832052 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a3ff4-98bc-44a8-85b5-9300bf7d4acb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:23.833528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a3ff4-98bc-44a8-85b5-9300bf7d4acb service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:23.842880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1292/2572] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:27.687353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49fccc-26f3-46f2-ad96-7104af771ac5 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:27.753237 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49fccc-26f3-46f2-ad96-7104af771ac5 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:27.757866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:27.758514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:27.758790 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:27.759045 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] PATH_INFO: `/v3/users/821b1410a2054952812b89792c38d937` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:27.760322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=821b1410a2054952812b89792c38d937)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:27.762188 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:27.816508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '821b1410a2054952812b89792c38d937'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.817205 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '821b1410a2054952812b89792c38d937'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.817824 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '821b1410a2054952812b89792c38d937'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.830569 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '821b1410a2054952812b89792c38d937'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.840840 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '821b1410a2054952812b89792c38d937'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.854010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1281/2573] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:27 2022] DELETE /identity/v3/users/821b1410a2054952812b89792c38d937 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:27.861190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87da0a-9b5a-4604-af1a-a7981104f59c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:27.862129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87da0a-9b5a-4604-af1a-a7981104f59c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:27.862684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87da0a-9b5a-4604-af1a-a7981104f59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:27.885879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-5a87da0a-9b5a-4604-af1a-a7981104f59c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:27.887376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5a87da0a-9b5a-4604-af1a-a7981104f59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:27.944773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:27.963604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1293/2574] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:27.972905 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:27.978120 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:28.057638 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77b13d-061f-4ba5-904b-2f4df8b2ae04 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:28.061572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bad7d-b762-4be0-8d2f-724d656dd598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:28.062244 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bad7d-b762-4be0-8d2f-724d656dd598 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.062590 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bad7d-b762-4be0-8d2f-724d656dd598 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.062930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bad7d-b762-4be0-8d2f-724d656dd598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.072483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bad7d-b762-4be0-8d2f-724d656dd598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:28.073948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bad7d-b762-4be0-8d2f-724d656dd598 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:28.083806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1282/2575] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:28.367608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87da0a-9b5a-4604-af1a-a7981104f59c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:28.369418 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5a87da0a-9b5a-4604-af1a-a7981104f59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:28.381654 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87da0a-9b5a-4604-af1a-a7981104f59c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:28.387023 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:28.387668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.387914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.388506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] PATH_INFO: `/v3/projects/9b3ecd2d2d064faabb5807733e72b917` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.389696 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b3ecd2d2d064faabb5807733e72b917)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:28.391598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:28.443207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b3ecd2d2d064faabb5807733e72b917', 'request_id': 'req-c7fc2a02-4957-405f-b37e-6e01c7be767b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:28.451986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b3ecd2d2d064faabb5807733e72b917', 'request_id': 'req-c7fc2a02-4957-405f-b37e-6e01c7be767b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:28.458793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b3ecd2d2d064faabb5807733e72b917', 'request_id': 'req-c7fc2a02-4957-405f-b37e-6e01c7be767b'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:28.469920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] The token cache is being invalidate because project 9b3ecd2d2d064faabb5807733e72b917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:28.470209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:28.470866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:28.472548 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1294/2576] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:28 2022] DELETE /identity/v3/projects/9b3ecd2d2d064faabb5807733e72b917 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:28.476628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bad7d-b762-4be0-8d2f-724d656dd598 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:28.487528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bad7d-b762-4be0-8d2f-724d656dd598 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:28.500988 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:28.553528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da294084-5b9e-4b69-94dc-1dbe2da04268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:28.554282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da294084-5b9e-4b69-94dc-1dbe2da04268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.554562 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da294084-5b9e-4b69-94dc-1dbe2da04268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.554835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da294084-5b9e-4b69-94dc-1dbe2da04268 admin admin] PATH_INFO: `/v3/users/7d960d714755477cbf5351531c7d5e4a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.556016 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da294084-5b9e-4b69-94dc-1dbe2da04268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d960d714755477cbf5351531c7d5e4a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:28.558666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da294084-5b9e-4b69-94dc-1dbe2da04268 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:28.611195 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-da294084-5b9e-4b69-94dc-1dbe2da04268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d960d714755477cbf5351531c7d5e4a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:28.612086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-da294084-5b9e-4b69-94dc-1dbe2da04268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d960d714755477cbf5351531c7d5e4a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:28.612693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-da294084-5b9e-4b69-94dc-1dbe2da04268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d960d714755477cbf5351531c7d5e4a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:28.626192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-da294084-5b9e-4b69-94dc-1dbe2da04268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d960d714755477cbf5351531c7d5e4a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:28.636880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-da294084-5b9e-4b69-94dc-1dbe2da04268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d960d714755477cbf5351531c7d5e4a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:28.651247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1283/2577] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:28 2022] DELETE /identity/v3/users/7d960d714755477cbf5351531c7d5e4a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:28.938154 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:28.939917 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:28.954615 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc2a02-4957-405f-b37e-6e01c7be767b admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:29.021500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59ff7d-d837-4900-b845-e53359d2506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:29.022093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59ff7d-d837-4900-b845-e53359d2506a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:29.022377 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59ff7d-d837-4900-b845-e53359d2506a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:29.022650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59ff7d-d837-4900-b845-e53359d2506a admin admin] PATH_INFO: `/v3/projects/17ff3395e0bd4380aca56f9b3ad4fd3c` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:29.023777 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59ff7d-d837-4900-b845-e53359d2506a admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ff3395e0bd4380aca56f9b3ad4fd3c)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:29.026369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59ff7d-d837-4900-b845-e53359d2506a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:29.083110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fb59ff7d-d837-4900-b845-e53359d2506a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ff3395e0bd4380aca56f9b3ad4fd3c', 'request_id': 'req-fb59ff7d-d837-4900-b845-e53359d2506a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:29.097179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fb59ff7d-d837-4900-b845-e53359d2506a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ff3395e0bd4380aca56f9b3ad4fd3c', 'request_id': 'req-fb59ff7d-d837-4900-b845-e53359d2506a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:29.105945 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fb59ff7d-d837-4900-b845-e53359d2506a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ff3395e0bd4380aca56f9b3ad4fd3c', 'request_id': 'req-fb59ff7d-d837-4900-b845-e53359d2506a'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:29.121992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fb59ff7d-d837-4900-b845-e53359d2506a admin admin] The token cache is being invalidate because project 17ff3395e0bd4380aca56f9b3ad4fd3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:29.122298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fb59ff7d-d837-4900-b845-e53359d2506a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:29.122994 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-fb59ff7d-d837-4900-b845-e53359d2506a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:29.125050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1295/2578] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:28 2022] DELETE /identity/v3/projects/17ff3395e0bd4380aca56f9b3ad4fd3c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:29.130185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccd724-888c-4ef9-83b0-2b966efbb63a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:29.130659 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccd724-888c-4ef9-83b0-2b966efbb63a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:29.130843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccd724-888c-4ef9-83b0-2b966efbb63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:29.152715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-80ccd724-888c-4ef9-83b0-2b966efbb63a None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:29.153933 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-80ccd724-888c-4ef9-83b0-2b966efbb63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:29.237358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1284/2579] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:29.239128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5361448-a151-421d-99e7-74bc60997ea8 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:29.239839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5361448-a151-421d-99e7-74bc60997ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:29.239839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5361448-a151-421d-99e7-74bc60997ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:29.256959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:29.263288 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-e5361448-a151-421d-99e7-74bc60997ea8 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:29.265459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e5361448-a151-421d-99e7-74bc60997ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:29.278574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1296/2580] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:29.279479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccd724-888c-4ef9-83b0-2b966efbb63a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:29.289198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccd724-888c-4ef9-83b0-2b966efbb63a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:29.298888 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e79469-24d0-4f5b-9690-59b2ad952fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:29.299684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e79469-24d0-4f5b-9690-59b2ad952fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:29.300043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e79469-24d0-4f5b-9690-59b2ad952fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:29.300411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e79469-24d0-4f5b-9690-59b2ad952fb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:29.301219 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e79469-24d0-4f5b-9690-59b2ad952fb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:29.301731 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e79469-24d0-4f5b-9690-59b2ad952fb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:29.303467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e79469-24d0-4f5b-9690-59b2ad952fb2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:29.316251 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1285/2581] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:29.320702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5361448-a151-421d-99e7-74bc60997ea8 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:29.330768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5361448-a151-421d-99e7-74bc60997ea8 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:29.335684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70000d65-4808-4558-8949-6e46a2433ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:29.336374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70000d65-4808-4558-8949-6e46a2433ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:29.336860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70000d65-4808-4558-8949-6e46a2433ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:29.337218 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70000d65-4808-4558-8949-6e46a2433ce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:29.338070 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70000d65-4808-4558-8949-6e46a2433ce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:29.340204 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70000d65-4808-4558-8949-6e46a2433ce2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:29.381330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1297/2582] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:29 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:29.385593 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e79469-24d0-4f5b-9690-59b2ad952fb2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:29.397270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e79469-24d0-4f5b-9690-59b2ad952fb2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:29.423802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:29.474602 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cec277-71a2-4557-b031-04ab759321d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:29.475219 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cec277-71a2-4557-b031-04ab759321d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:29.475495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cec277-71a2-4557-b031-04ab759321d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:29.475767 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cec277-71a2-4557-b031-04ab759321d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:29.476401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cec277-71a2-4557-b031-04ab759321d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:29.478892 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cec277-71a2-4557-b031-04ab759321d4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:29.508967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1286/2583] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:29 2022] POST /identity/v3/users => generated 285 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:29.510902 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70000d65-4808-4558-8949-6e46a2433ce2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:29.522483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70000d65-4808-4558-8949-6e46a2433ce2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:29.528365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ee0a8-1b3c-497a-91a3-2ea6bec0e102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:29.529107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ee0a8-1b3c-497a-91a3-2ea6bec0e102 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:29.529328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ee0a8-1b3c-497a-91a3-2ea6bec0e102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:29.529603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ee0a8-1b3c-497a-91a3-2ea6bec0e102 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:29.530269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ee0a8-1b3c-497a-91a3-2ea6bec0e102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:29.531921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ee0a8-1b3c-497a-91a3-2ea6bec0e102 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:29.544199 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1298/2584] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:29.548695 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cec277-71a2-4557-b031-04ab759321d4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:29.560509 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cec277-71a2-4557-b031-04ab759321d4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:29.566128 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df17b8-1f44-49a0-bafd-fbd77f22563d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:29.566740 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df17b8-1f44-49a0-bafd-fbd77f22563d admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:29.567023 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df17b8-1f44-49a0-bafd-fbd77f22563d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:29.567304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df17b8-1f44-49a0-bafd-fbd77f22563d admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:29.568008 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df17b8-1f44-49a0-bafd-fbd77f22563d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:29.569822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df17b8-1f44-49a0-bafd-fbd77f22563d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:29.583421 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1287/2585] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:29.587652 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ee0a8-1b3c-497a-91a3-2ea6bec0e102 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:29.599920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ee0a8-1b3c-497a-91a3-2ea6bec0e102 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:29.606291 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72304974-7d14-4e3f-af24-38ddd215c8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:29.607136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72304974-7d14-4e3f-af24-38ddd215c8ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:29.607530 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72304974-7d14-4e3f-af24-38ddd215c8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:29.608213 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72304974-7d14-4e3f-af24-38ddd215c8ab admin admin] PATH_INFO: `/v3/projects/5e08ea14f2cd48b1990b43e57d3bbde5/users/713450d09cec49f8beaa588b702e4844/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:29.627316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72304974-7d14-4e3f-af24-38ddd215c8ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e08ea14f2cd48b1990b43e57d3bbde5, user_id=713450d09cec49f8beaa588b702e4844, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:29.630246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72304974-7d14-4e3f-af24-38ddd215c8ab admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:29.647986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1299/2586] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:29 2022] PUT /identity/v3/projects/5e08ea14f2cd48b1990b43e57d3bbde5/users/713450d09cec49f8beaa588b702e4844/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:29.663075 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6d768-e20c-4db1-a1c4-fcfadfc96445 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:29.663329 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6d768-e20c-4db1-a1c4-fcfadfc96445 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:29.663679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6d768-e20c-4db1-a1c4-fcfadfc96445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:29.698967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f5c6d768-e20c-4db1-a1c4-fcfadfc96445 None None] MFA Rules not processed for user `713450d09cec49f8beaa588b702e4844`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:29.700415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f5c6d768-e20c-4db1-a1c4-fcfadfc96445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:29.809270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1288/2587] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:29.813851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72304974-7d14-4e3f-af24-38ddd215c8ab admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:29.882795 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72304974-7d14-4e3f-af24-38ddd215c8ab admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:29.888034 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e7517-4497-4e57-8c48-267b7e296a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:29.888797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e7517-4497-4e57-8c48-267b7e296a29 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:29.889137 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e7517-4497-4e57-8c48-267b7e296a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:29.889472 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e7517-4497-4e57-8c48-267b7e296a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:29.890173 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e7517-4497-4e57-8c48-267b7e296a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:29.892156 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e7517-4497-4e57-8c48-267b7e296a29 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:29.933333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1300/2588] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:29 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:29.937446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6d768-e20c-4db1-a1c4-fcfadfc96445 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:29.948548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6d768-e20c-4db1-a1c4-fcfadfc96445 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:30.010860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5d3a6-6418-4ba3-9dc9-e99c5ce42145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:30.011467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5d3a6-6418-4ba3-9dc9-e99c5ce42145 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:30.011893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5d3a6-6418-4ba3-9dc9-e99c5ce42145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:30.012167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5d3a6-6418-4ba3-9dc9-e99c5ce42145 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:30.012850 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5d3a6-6418-4ba3-9dc9-e99c5ce42145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:30.014802 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5d3a6-6418-4ba3-9dc9-e99c5ce42145 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:30.037747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1289/2589] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:29 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:30.042642 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e7517-4497-4e57-8c48-267b7e296a29 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:30.053362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e7517-4497-4e57-8c48-267b7e296a29 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:30.058413 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7700e00-a1c6-46a4-873a-07b21776124a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:30.059111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7700e00-a1c6-46a4-873a-07b21776124a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:30.059446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7700e00-a1c6-46a4-873a-07b21776124a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:30.059772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7700e00-a1c6-46a4-873a-07b21776124a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:30.060473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7700e00-a1c6-46a4-873a-07b21776124a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:30.062179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7700e00-a1c6-46a4-873a-07b21776124a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:30.074687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1301/2590] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:30.079286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5d3a6-6418-4ba3-9dc9-e99c5ce42145 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:30.089853 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5d3a6-6418-4ba3-9dc9-e99c5ce42145 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:30.094773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b0a96-5bbd-40fa-88d4-3b0a55bafc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:30.095446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b0a96-5bbd-40fa-88d4-3b0a55bafc80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:30.095727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b0a96-5bbd-40fa-88d4-3b0a55bafc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:30.095997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b0a96-5bbd-40fa-88d4-3b0a55bafc80 admin admin] PATH_INFO: `/v3/projects/fe5be6ce389040fe827f6c3734b551eb/users/426ca0ea10fb4291b1c034bb8b6f306e/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:30.113615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b0a96-5bbd-40fa-88d4-3b0a55bafc80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5be6ce389040fe827f6c3734b551eb, user_id=426ca0ea10fb4291b1c034bb8b6f306e, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:30.116331 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b0a96-5bbd-40fa-88d4-3b0a55bafc80 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:30.119778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:30.134827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1290/2591] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:30 2022] PUT /identity/v3/projects/fe5be6ce389040fe827f6c3734b551eb/users/426ca0ea10fb4291b1c034bb8b6f306e/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:30.139232 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7700e00-a1c6-46a4-873a-07b21776124a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:30.151208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7700e00-a1c6-46a4-873a-07b21776124a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:30.211206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7d6ba-1a3e-43c4-a9c7-5d8a6b00519c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:30.211816 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7d6ba-1a3e-43c4-a9c7-5d8a6b00519c admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:30.212092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7d6ba-1a3e-43c4-a9c7-5d8a6b00519c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:30.212360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7d6ba-1a3e-43c4-a9c7-5d8a6b00519c admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:30.213112 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7d6ba-1a3e-43c4-a9c7-5d8a6b00519c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:30.215080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7d6ba-1a3e-43c4-a9c7-5d8a6b00519c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:30.230769 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1302/2592] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:30.235416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b0a96-5bbd-40fa-88d4-3b0a55bafc80 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:30.248636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b0a96-5bbd-40fa-88d4-3b0a55bafc80 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:30.254014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7a814-73ab-4689-849f-476db9190233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:30.254822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7a814-73ab-4689-849f-476db9190233 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:30.255203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7a814-73ab-4689-849f-476db9190233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:30.255536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7a814-73ab-4689-849f-476db9190233 admin admin] PATH_INFO: `/v3/projects/fe5be6ce389040fe827f6c3734b551eb/users/426ca0ea10fb4291b1c034bb8b6f306e/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:30.257755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7a814-73ab-4689-849f-476db9190233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5be6ce389040fe827f6c3734b551eb, user_id=426ca0ea10fb4291b1c034bb8b6f306e, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:30.261131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7a814-73ab-4689-849f-476db9190233 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:30.274949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1291/2593] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:30 2022] PUT /identity/v3/projects/fe5be6ce389040fe827f6c3734b551eb/users/426ca0ea10fb4291b1c034bb8b6f306e/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:30.285733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5835c-efcd-4bcc-9b48-c572a86b49e4 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:30.285733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5835c-efcd-4bcc-9b48-c572a86b49e4 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:30.286099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5835c-efcd-4bcc-9b48-c572a86b49e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:30.316590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:30.320996 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-bfc5835c-efcd-4bcc-9b48-c572a86b49e4 None None] MFA Rules not processed for user `426ca0ea10fb4291b1c034bb8b6f306e`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:30.323427 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bfc5835c-efcd-4bcc-9b48-c572a86b49e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:30.420923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1303/2594] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:30.433114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7a814-73ab-4689-849f-476db9190233 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:30.434934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6bc7a814-73ab-4689-849f-476db9190233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:30.510132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7a814-73ab-4689-849f-476db9190233 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:30.514309 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a94432-e827-4d05-9b42-1db6442697ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:30.515064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a94432-e827-4d05-9b42-1db6442697ff service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:30.515411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a94432-e827-4d05-9b42-1db6442697ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:30.515767 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a94432-e827-4d05-9b42-1db6442697ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:30.526579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a94432-e827-4d05-9b42-1db6442697ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:30.528296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a94432-e827-4d05-9b42-1db6442697ff service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:30.548285 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1292/2595] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:30.601905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5835c-efcd-4bcc-9b48-c572a86b49e4 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:30.613301 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5835c-efcd-4bcc-9b48-c572a86b49e4 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:30.618153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfb549-15a0-4592-8c40-ab2eed06aafd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:30.620003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfb549-15a0-4592-8c40-ab2eed06aafd service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:30.620381 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfb549-15a0-4592-8c40-ab2eed06aafd service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:30.621391 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfb549-15a0-4592-8c40-ab2eed06aafd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:30.669732 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a94432-e827-4d05-9b42-1db6442697ff service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:30.671183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-02a94432-e827-4d05-9b42-1db6442697ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:30.696078 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfb549-15a0-4592-8c40-ab2eed06aafd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:30.697931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfb549-15a0-4592-8c40-ab2eed06aafd service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:30.708103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1304/2596] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:30.751781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a94432-e827-4d05-9b42-1db6442697ff service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:30.757718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:30.758387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:30.758659 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:30.758935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] PATH_INFO: `/v3/users/105862091d7344118c1505d6f6b01f85` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:30.760553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=105862091d7344118c1505d6f6b01f85)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:30.762552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:30.820832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '105862091d7344118c1505d6f6b01f85'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:30.821550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '105862091d7344118c1505d6f6b01f85'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:30.823089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '105862091d7344118c1505d6f6b01f85'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:30.842060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '105862091d7344118c1505d6f6b01f85'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:30.852428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '105862091d7344118c1505d6f6b01f85'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:30.867996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1293/2597] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:30 2022] DELETE /identity/v3/users/105862091d7344118c1505d6f6b01f85 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:30.872320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026fac4-623c-4191-b2c6-eec8d7f85abd None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:30.872819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026fac4-623c-4191-b2c6-eec8d7f85abd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:30.873099 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026fac4-623c-4191-b2c6-eec8d7f85abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:30.901331 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a026fac4-623c-4191-b2c6-eec8d7f85abd None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:30.902752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a026fac4-623c-4191-b2c6-eec8d7f85abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:30.971359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1305/2598] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:30.982977 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:30.984781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:30.993409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:31.059859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914649fd-9d4f-49dc-9bfe-3f6a5ae54cb7 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:31.063732 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275d0d6-4814-4390-8161-06e9142131a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:31.064341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275d0d6-4814-4390-8161-06e9142131a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:31.064617 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275d0d6-4814-4390-8161-06e9142131a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:31.064932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275d0d6-4814-4390-8161-06e9142131a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:31.074811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275d0d6-4814-4390-8161-06e9142131a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:31.076229 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275d0d6-4814-4390-8161-06e9142131a5 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:31.086190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1294/2599] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:31.381064 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026fac4-623c-4191-b2c6-eec8d7f85abd None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:31.382617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a026fac4-623c-4191-b2c6-eec8d7f85abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:31.395722 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026fac4-623c-4191-b2c6-eec8d7f85abd None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:31.402919 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:31.403896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:31.404168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:31.404430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] PATH_INFO: `/v3/projects/a9000c16354a4405b498a5b2a228b905` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:31.405594 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9000c16354a4405b498a5b2a228b905)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:31.408056 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:31.461168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:31.474380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9000c16354a4405b498a5b2a228b905', 'request_id': 'req-04751adc-aa7a-4a20-b315-cb08c152fbaf'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:31.482546 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9000c16354a4405b498a5b2a228b905', 'request_id': 'req-04751adc-aa7a-4a20-b315-cb08c152fbaf'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:31.489549 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9000c16354a4405b498a5b2a228b905', 'request_id': 'req-04751adc-aa7a-4a20-b315-cb08c152fbaf'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:31.500197 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] The token cache is being invalidate because project a9000c16354a4405b498a5b2a228b905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:31.500482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:31.501184 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:31.502703 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1306/2600] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:31 2022] DELETE /identity/v3/projects/a9000c16354a4405b498a5b2a228b905 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:31.506891 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275d0d6-4814-4390-8161-06e9142131a5 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:31.516755 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275d0d6-4814-4390-8161-06e9142131a5 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:31.582178 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4db71b-a11e-4727-96e6-b9b7c08f7201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:31.582820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4db71b-a11e-4727-96e6-b9b7c08f7201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:31.583098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4db71b-a11e-4727-96e6-b9b7c08f7201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:31.583382 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4db71b-a11e-4727-96e6-b9b7c08f7201 admin admin] PATH_INFO: `/v3/users/fd56de52c58f4f369a426d94d759e294` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:31.584622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4db71b-a11e-4727-96e6-b9b7c08f7201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd56de52c58f4f369a426d94d759e294)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:31.586607 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4db71b-a11e-4727-96e6-b9b7c08f7201 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:31.648545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1e4db71b-a11e-4727-96e6-b9b7c08f7201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd56de52c58f4f369a426d94d759e294'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:31.649567 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1e4db71b-a11e-4727-96e6-b9b7c08f7201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd56de52c58f4f369a426d94d759e294'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:31.650654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1e4db71b-a11e-4727-96e6-b9b7c08f7201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd56de52c58f4f369a426d94d759e294'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:31.670642 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1e4db71b-a11e-4727-96e6-b9b7c08f7201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd56de52c58f4f369a426d94d759e294'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:31.683100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-1e4db71b-a11e-4727-96e6-b9b7c08f7201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd56de52c58f4f369a426d94d759e294'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:31.700137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1295/2601] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:31 2022] DELETE /identity/v3/users/fd56de52c58f4f369a426d94d759e294 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:31.980077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:31.984101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:32.001485 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04751adc-aa7a-4a20-b315-cb08c152fbaf admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:32.077371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf54631-d631-4546-9982-a08b0eb89bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:32.078501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf54631-d631-4546-9982-a08b0eb89bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:32.078866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf54631-d631-4546-9982-a08b0eb89bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:32.079172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf54631-d631-4546-9982-a08b0eb89bc1 admin admin] PATH_INFO: `/v3/projects/f2fdef372eb34b47a07facf95eaa4768` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:32.080473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf54631-d631-4546-9982-a08b0eb89bc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2fdef372eb34b47a07facf95eaa4768)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:32.082749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf54631-d631-4546-9982-a08b0eb89bc1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:32.152752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7bf54631-d631-4546-9982-a08b0eb89bc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2fdef372eb34b47a07facf95eaa4768', 'request_id': 'req-7bf54631-d631-4546-9982-a08b0eb89bc1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:32.164407 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7bf54631-d631-4546-9982-a08b0eb89bc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2fdef372eb34b47a07facf95eaa4768', 'request_id': 'req-7bf54631-d631-4546-9982-a08b0eb89bc1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:32.174192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7bf54631-d631-4546-9982-a08b0eb89bc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2fdef372eb34b47a07facf95eaa4768', 'request_id': 'req-7bf54631-d631-4546-9982-a08b0eb89bc1'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:32.194582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7bf54631-d631-4546-9982-a08b0eb89bc1 admin admin] The token cache is being invalidate because project f2fdef372eb34b47a07facf95eaa4768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:32.194948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7bf54631-d631-4546-9982-a08b0eb89bc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:32.195668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7bf54631-d631-4546-9982-a08b0eb89bc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:32.197783 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1307/2602] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:31 2022] DELETE /identity/v3/projects/f2fdef372eb34b47a07facf95eaa4768 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:32.203172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8772207-b47b-4c45-ba0c-38c58022f0f5 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:32.203761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8772207-b47b-4c45-ba0c-38c58022f0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:32.204048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8772207-b47b-4c45-ba0c-38c58022f0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:32.229304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-f8772207-b47b-4c45-ba0c-38c58022f0f5 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:32.230678 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f8772207-b47b-4c45-ba0c-38c58022f0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:32.258393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:32.311568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1296/2603] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:32.319908 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e41daf-f521-4317-b679-14ca231523ec None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:32.320199 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e41daf-f521-4317-b679-14ca231523ec None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:32.320518 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e41daf-f521-4317-b679-14ca231523ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:32.341792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-06e41daf-f521-4317-b679-14ca231523ec None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:32.343020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-06e41daf-f521-4317-b679-14ca231523ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:32.351850 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1308/2604] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:32.356679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8772207-b47b-4c45-ba0c-38c58022f0f5 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:32.366772 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8772207-b47b-4c45-ba0c-38c58022f0f5 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:32.372385 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a928cb3-2162-42cb-b4f9-0baf503de7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:32.373017 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a928cb3-2162-42cb-b4f9-0baf503de7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:32.373429 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a928cb3-2162-42cb-b4f9-0baf503de7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:32.373714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a928cb3-2162-42cb-b4f9-0baf503de7f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:32.374468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a928cb3-2162-42cb-b4f9-0baf503de7f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:32.374773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a928cb3-2162-42cb-b4f9-0baf503de7f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:32.376625 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a928cb3-2162-42cb-b4f9-0baf503de7f4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:32.392746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1297/2605] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:32.395166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e41daf-f521-4317-b679-14ca231523ec None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:32.405086 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e41daf-f521-4317-b679-14ca231523ec None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:32.409969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b2ff7-5f91-4158-b4ce-fe5620d0706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:32.410556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b2ff7-5f91-4158-b4ce-fe5620d0706c admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:32.411016 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b2ff7-5f91-4158-b4ce-fe5620d0706c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:32.411347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b2ff7-5f91-4158-b4ce-fe5620d0706c admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:32.412066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b2ff7-5f91-4158-b4ce-fe5620d0706c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:32.414102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b2ff7-5f91-4158-b4ce-fe5620d0706c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:32.443113 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1309/2606] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:32 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:32.447712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a928cb3-2162-42cb-b4f9-0baf503de7f4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:32.459726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a928cb3-2162-42cb-b4f9-0baf503de7f4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:32.528350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a5d4b-20c0-461b-a1f5-5119aa76ae93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:32.528930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a5d4b-20c0-461b-a1f5-5119aa76ae93 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:32.528930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a5d4b-20c0-461b-a1f5-5119aa76ae93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:32.529249 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a5d4b-20c0-461b-a1f5-5119aa76ae93 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:32.529780 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a5d4b-20c0-461b-a1f5-5119aa76ae93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:32.532343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a5d4b-20c0-461b-a1f5-5119aa76ae93 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:32.563325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1298/2607] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:32 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:32.567942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b2ff7-5f91-4158-b4ce-fe5620d0706c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:32.579439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b2ff7-5f91-4158-b4ce-fe5620d0706c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:32.584634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95f324-316f-4744-ae77-5e38cfffcb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:32.585297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95f324-316f-4744-ae77-5e38cfffcb1a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:32.585625 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95f324-316f-4744-ae77-5e38cfffcb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:32.585829 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95f324-316f-4744-ae77-5e38cfffcb1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:32.586482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95f324-316f-4744-ae77-5e38cfffcb1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:32.588153 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95f324-316f-4744-ae77-5e38cfffcb1a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:32.599124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1310/2608] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:32.603821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a5d4b-20c0-461b-a1f5-5119aa76ae93 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:32.616164 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a5d4b-20c0-461b-a1f5-5119aa76ae93 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:32.621438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a16ed3-ba04-40db-95fc-e1727163d6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:32.622043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a16ed3-ba04-40db-95fc-e1727163d6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:32.622337 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a16ed3-ba04-40db-95fc-e1727163d6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:32.622601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a16ed3-ba04-40db-95fc-e1727163d6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:32.623288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a16ed3-ba04-40db-95fc-e1727163d6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:32.625241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a16ed3-ba04-40db-95fc-e1727163d6c6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:32.637192 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1299/2609] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:32.641659 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95f324-316f-4744-ae77-5e38cfffcb1a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:32.651960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95f324-316f-4744-ae77-5e38cfffcb1a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:32.656477 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb92044-75eb-48ec-a271-54ae9ba46f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:32.657166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb92044-75eb-48ec-a271-54ae9ba46f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:32.657446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb92044-75eb-48ec-a271-54ae9ba46f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:32.657719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb92044-75eb-48ec-a271-54ae9ba46f96 admin admin] PATH_INFO: `/v3/projects/12abf7cfb7cb479a9ffa5d0a4c73b468/users/d52d21142c49425dbc81565e9f31ca15/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:32.668521 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:32.676657 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb92044-75eb-48ec-a271-54ae9ba46f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12abf7cfb7cb479a9ffa5d0a4c73b468, user_id=d52d21142c49425dbc81565e9f31ca15, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:32.679813 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb92044-75eb-48ec-a271-54ae9ba46f96 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:32.696504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1311/2610] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:32 2022] PUT /identity/v3/projects/12abf7cfb7cb479a9ffa5d0a4c73b468/users/d52d21142c49425dbc81565e9f31ca15/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:32.707523 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282044d-8f9e-4f30-97fc-79230185d6fb None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:32.707803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282044d-8f9e-4f30-97fc-79230185d6fb None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:32.708067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282044d-8f9e-4f30-97fc-79230185d6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:32.738707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a282044d-8f9e-4f30-97fc-79230185d6fb None None] MFA Rules not processed for user `d52d21142c49425dbc81565e9f31ca15`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:32.739979 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a282044d-8f9e-4f30-97fc-79230185d6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:32.808231 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1300/2611] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:32.813093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb92044-75eb-48ec-a271-54ae9ba46f96 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:32.877011 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb92044-75eb-48ec-a271-54ae9ba46f96 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:32.883314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec62499-0927-4ed3-9c1b-233a4d2a31a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:32.883927 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec62499-0927-4ed3-9c1b-233a4d2a31a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:32.884305 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec62499-0927-4ed3-9c1b-233a4d2a31a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:32.884979 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec62499-0927-4ed3-9c1b-233a4d2a31a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:32.885643 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec62499-0927-4ed3-9c1b-233a4d2a31a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:32.887576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec62499-0927-4ed3-9c1b-233a4d2a31a1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:32.919866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1312/2612] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:32 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:32.926247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282044d-8f9e-4f30-97fc-79230185d6fb None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:32.940141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282044d-8f9e-4f30-97fc-79230185d6fb None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:32.962907 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:33.007380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22ecf0-da15-4a91-a629-9537f40abf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:33.008021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22ecf0-da15-4a91-a629-9537f40abf30 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:33.008317 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22ecf0-da15-4a91-a629-9537f40abf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:33.008585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22ecf0-da15-4a91-a629-9537f40abf30 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:33.009256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22ecf0-da15-4a91-a629-9537f40abf30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:33.011378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22ecf0-da15-4a91-a629-9537f40abf30 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:33.035060 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1301/2613] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:32 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:33.039312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec62499-0927-4ed3-9c1b-233a4d2a31a1 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:33.050641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec62499-0927-4ed3-9c1b-233a4d2a31a1 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:33.056317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5218f-a3ca-4485-8105-7998a10bd267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:33.057053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5218f-a3ca-4485-8105-7998a10bd267 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:33.057333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5218f-a3ca-4485-8105-7998a10bd267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:33.057603 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5218f-a3ca-4485-8105-7998a10bd267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:33.058255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5218f-a3ca-4485-8105-7998a10bd267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:33.059907 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5218f-a3ca-4485-8105-7998a10bd267 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:33.071103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1313/2614] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:33.075935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22ecf0-da15-4a91-a629-9537f40abf30 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:33.086341 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22ecf0-da15-4a91-a629-9537f40abf30 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:33.091965 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9241a-c95c-4e5a-a319-fe0daec25056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:33.092634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9241a-c95c-4e5a-a319-fe0daec25056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:33.092942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9241a-c95c-4e5a-a319-fe0daec25056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:33.093214 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9241a-c95c-4e5a-a319-fe0daec25056 admin admin] PATH_INFO: `/v3/projects/591bdd6cf5f4479d9a00e43822ef3255/users/71379027edf14a889da6e50978a8a41c/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:33.110475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9241a-c95c-4e5a-a319-fe0daec25056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=591bdd6cf5f4479d9a00e43822ef3255, user_id=71379027edf14a889da6e50978a8a41c, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:33.113280 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9241a-c95c-4e5a-a319-fe0daec25056 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:33.130097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1302/2615] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:33 2022] PUT /identity/v3/projects/591bdd6cf5f4479d9a00e43822ef3255/users/71379027edf14a889da6e50978a8a41c/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:33.134655 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5218f-a3ca-4485-8105-7998a10bd267 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:33.145212 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5218f-a3ca-4485-8105-7998a10bd267 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:33.212318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556de140-bde5-409a-ba95-997dff0f22a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:33.212978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556de140-bde5-409a-ba95-997dff0f22a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:33.213265 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556de140-bde5-409a-ba95-997dff0f22a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:33.213524 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556de140-bde5-409a-ba95-997dff0f22a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:33.214172 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556de140-bde5-409a-ba95-997dff0f22a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:33.215756 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556de140-bde5-409a-ba95-997dff0f22a0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:33.225943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1314/2616] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:33.232136 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9241a-c95c-4e5a-a319-fe0daec25056 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:33.247986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9241a-c95c-4e5a-a319-fe0daec25056 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:33.253527 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100bf8a-fdea-4851-b113-ce0a98cafa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:33.254177 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100bf8a-fdea-4851-b113-ce0a98cafa5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:33.254456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100bf8a-fdea-4851-b113-ce0a98cafa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:33.254854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100bf8a-fdea-4851-b113-ce0a98cafa5e admin admin] PATH_INFO: `/v3/projects/591bdd6cf5f4479d9a00e43822ef3255/users/71379027edf14a889da6e50978a8a41c/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:33.256894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100bf8a-fdea-4851-b113-ce0a98cafa5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=591bdd6cf5f4479d9a00e43822ef3255, user_id=71379027edf14a889da6e50978a8a41c, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:33.260100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100bf8a-fdea-4851-b113-ce0a98cafa5e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:33.271239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1303/2617] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:33 2022] PUT /identity/v3/projects/591bdd6cf5f4479d9a00e43822ef3255/users/71379027edf14a889da6e50978a8a41c/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:33.282779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1edb8-dec5-430a-b75a-47b42e044bd1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:33.283061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1edb8-dec5-430a-b75a-47b42e044bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:33.283352 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1edb8-dec5-430a-b75a-47b42e044bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:33.312763 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-faa1edb8-dec5-430a-b75a-47b42e044bd1 None None] MFA Rules not processed for user `71379027edf14a889da6e50978a8a41c`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:33.314467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-faa1edb8-dec5-430a-b75a-47b42e044bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:33.363204 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:33.394541 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1315/2618] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:33.409680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100bf8a-fdea-4851-b113-ce0a98cafa5e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:33.415910 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0100bf8a-fdea-4851-b113-ce0a98cafa5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:33.490796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100bf8a-fdea-4851-b113-ce0a98cafa5e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:33.494822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0851131-492a-4634-b315-58d7cd2a397d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:33.495498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0851131-492a-4634-b315-58d7cd2a397d service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:33.495831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0851131-492a-4634-b315-58d7cd2a397d service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:33.496100 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0851131-492a-4634-b315-58d7cd2a397d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:33.505738 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0851131-492a-4634-b315-58d7cd2a397d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:33.507380 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0851131-492a-4634-b315-58d7cd2a397d service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:33.518022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1304/2619] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:34.157856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1edb8-dec5-430a-b75a-47b42e044bd1 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:34.160307 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-faa1edb8-dec5-430a-b75a-47b42e044bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:34.234392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1edb8-dec5-430a-b75a-47b42e044bd1 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:34.239328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:34.240148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:34.240523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:34.240916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] PATH_INFO: `/v3/users/713450d09cec49f8beaa588b702e4844` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:34.242163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=713450d09cec49f8beaa588b702e4844)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:34.244971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:34.302825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '713450d09cec49f8beaa588b702e4844'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:34.303509 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '713450d09cec49f8beaa588b702e4844'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:34.304115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '713450d09cec49f8beaa588b702e4844'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:34.317896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '713450d09cec49f8beaa588b702e4844'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:34.328130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '713450d09cec49f8beaa588b702e4844'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:34.341927 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1316/2620] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:34 2022] DELETE /identity/v3/users/713450d09cec49f8beaa588b702e4844 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:34.346429 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707f7e8-6979-48d0-9b47-57c34562929f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:34.347843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707f7e8-6979-48d0-9b47-57c34562929f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:34.348121 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707f7e8-6979-48d0-9b47-57c34562929f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:34.369462 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5707f7e8-6979-48d0-9b47-57c34562929f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:34.370699 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5707f7e8-6979-48d0-9b47-57c34562929f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:34.436693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1305/2621] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:34.446273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:34.450225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:34.514319 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119bfc6-5390-4a9a-aa82-70fb59559d9f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:34.518156 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab2e11-846f-4afa-ac1c-57e032e0a60a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:34.518772 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab2e11-846f-4afa-ac1c-57e032e0a60a service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:34.519055 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab2e11-846f-4afa-ac1c-57e032e0a60a service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:34.519324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab2e11-846f-4afa-ac1c-57e032e0a60a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:34.529790 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab2e11-846f-4afa-ac1c-57e032e0a60a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:34.532806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab2e11-846f-4afa-ac1c-57e032e0a60a service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:34.542777 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1317/2622] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:34.652526 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707f7e8-6979-48d0-9b47-57c34562929f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:34.657106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5707f7e8-6979-48d0-9b47-57c34562929f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:34.671073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707f7e8-6979-48d0-9b47-57c34562929f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:34.678106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a38215-3914-4152-a5f2-ff363c7ebf05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:34.678721 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a38215-3914-4152-a5f2-ff363c7ebf05 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:34.678959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a38215-3914-4152-a5f2-ff363c7ebf05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:34.679555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a38215-3914-4152-a5f2-ff363c7ebf05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:34.682388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-13a38215-3914-4152-a5f2-ff363c7ebf05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:34.708984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a38215-3914-4152-a5f2-ff363c7ebf05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:34.710539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a38215-3914-4152-a5f2-ff363c7ebf05 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:34.722594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1306/2623] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:34.991028 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab2e11-846f-4afa-ac1c-57e032e0a60a service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:34.992869 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e8ab2e11-846f-4afa-ac1c-57e032e0a60a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:35.002978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:35.007310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab2e11-846f-4afa-ac1c-57e032e0a60a service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:35.014843 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5432216-ec2f-4732-8e3d-8f9e919f7a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:35.015658 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5432216-ec2f-4732-8e3d-8f9e919f7a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:35.016327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5432216-ec2f-4732-8e3d-8f9e919f7a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:35.016820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5432216-ec2f-4732-8e3d-8f9e919f7a17 admin admin] PATH_INFO: `/v3/users/d52d21142c49425dbc81565e9f31ca15` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:35.018341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5432216-ec2f-4732-8e3d-8f9e919f7a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52d21142c49425dbc81565e9f31ca15)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:35.020316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5432216-ec2f-4732-8e3d-8f9e919f7a17 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:35.038647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a38215-3914-4152-a5f2-ff363c7ebf05 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:35.041729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-13a38215-3914-4152-a5f2-ff363c7ebf05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:35.058406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:35.064867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a38215-3914-4152-a5f2-ff363c7ebf05 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:35.073342 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:35.075449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:35.076077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:35.077198 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] PATH_INFO: `/v3/projects/5e08ea14f2cd48b1990b43e57d3bbde5` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:35.079680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e08ea14f2cd48b1990b43e57d3bbde5)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:35.083228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:35.085804 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a5432216-ec2f-4732-8e3d-8f9e919f7a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52d21142c49425dbc81565e9f31ca15'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.086489 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a5432216-ec2f-4732-8e3d-8f9e919f7a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52d21142c49425dbc81565e9f31ca15'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.087527 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a5432216-ec2f-4732-8e3d-8f9e919f7a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52d21142c49425dbc81565e9f31ca15'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.103458 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a5432216-ec2f-4732-8e3d-8f9e919f7a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52d21142c49425dbc81565e9f31ca15'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.115570 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a5432216-ec2f-4732-8e3d-8f9e919f7a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52d21142c49425dbc81565e9f31ca15'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.132077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1318/2624] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:34 2022] DELETE /identity/v3/users/d52d21142c49425dbc81565e9f31ca15 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:35.137252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7c5d0-b1f2-43aa-b7c0-b0dc7271303c None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:35.137674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7c5d0-b1f2-43aa-b7c0-b0dc7271303c None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:35.137963 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7c5d0-b1f2-43aa-b7c0-b0dc7271303c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:35.150980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e08ea14f2cd48b1990b43e57d3bbde5', 'request_id': 'req-f46e206b-0641-4fc4-aa52-aa34da46d88b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.160584 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c6c7c5d0-b1f2-43aa-b7c0-b0dc7271303c None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:35.161737 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e08ea14f2cd48b1990b43e57d3bbde5', 'request_id': 'req-f46e206b-0641-4fc4-aa52-aa34da46d88b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.161880 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c6c7c5d0-b1f2-43aa-b7c0-b0dc7271303c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:35.169416 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e08ea14f2cd48b1990b43e57d3bbde5', 'request_id': 'req-f46e206b-0641-4fc4-aa52-aa34da46d88b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.182488 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] The token cache is being invalidate because project 5e08ea14f2cd48b1990b43e57d3bbde5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:35.182779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.183511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.185168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1307/2625] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:35 2022] DELETE /identity/v3/projects/5e08ea14f2cd48b1990b43e57d3bbde5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:35.189333 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:35.190766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:35.236580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1319/2626] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:35.248270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7c5d0-b1f2-43aa-b7c0-b0dc7271303c None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:35.250021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c6c7c5d0-b1f2-43aa-b7c0-b0dc7271303c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:35.264993 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e206b-0641-4fc4-aa52-aa34da46d88b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:35.269934 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:35.270579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:35.270859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:35.271131 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] PATH_INFO: `/v3/users/426ca0ea10fb4291b1c034bb8b6f306e` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:35.272305 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=426ca0ea10fb4291b1c034bb8b6f306e)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:35.274272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:35.317019 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7c5d0-b1f2-43aa-b7c0-b0dc7271303c None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:35.321881 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908e19b-12f7-4c6a-81e8-2d6dad40093c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:35.322871 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908e19b-12f7-4c6a-81e8-2d6dad40093c service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:35.323468 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908e19b-12f7-4c6a-81e8-2d6dad40093c service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:35.323890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908e19b-12f7-4c6a-81e8-2d6dad40093c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:35.334622 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908e19b-12f7-4c6a-81e8-2d6dad40093c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:35.335144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '426ca0ea10fb4291b1c034bb8b6f306e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.335968 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '426ca0ea10fb4291b1c034bb8b6f306e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.336384 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908e19b-12f7-4c6a-81e8-2d6dad40093c service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:35.336868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '426ca0ea10fb4291b1c034bb8b6f306e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.348223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1320/2627] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:40:35.352052 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '426ca0ea10fb4291b1c034bb8b6f306e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.362259 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '426ca0ea10fb4291b1c034bb8b6f306e'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.378158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1308/2628] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:35 2022] DELETE /identity/v3/users/426ca0ea10fb4291b1c034bb8b6f306e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:35.757813 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908e19b-12f7-4c6a-81e8-2d6dad40093c service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:35.759434 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-9908e19b-12f7-4c6a-81e8-2d6dad40093c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:35.773442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908e19b-12f7-4c6a-81e8-2d6dad40093c service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:35.793655 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:35.856737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:35.857487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:35.857833 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:35.858170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] PATH_INFO: `/v3/projects/12abf7cfb7cb479a9ffa5d0a4c73b468` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:35.860624 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12abf7cfb7cb479a9ffa5d0a4c73b468)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:35.862348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:35.931021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12abf7cfb7cb479a9ffa5d0a4c73b468', 'request_id': 'req-2df301cd-93e8-404d-8f5c-4e6f73212e68'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.942203 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12abf7cfb7cb479a9ffa5d0a4c73b468', 'request_id': 'req-2df301cd-93e8-404d-8f5c-4e6f73212e68'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.949799 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12abf7cfb7cb479a9ffa5d0a4c73b468', 'request_id': 'req-2df301cd-93e8-404d-8f5c-4e6f73212e68'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.978931 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] The token cache is being invalidate because project 12abf7cfb7cb479a9ffa5d0a4c73b468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:35.979461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.980344 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:35.984315 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1321/2629] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:35 2022] DELETE /identity/v3/projects/12abf7cfb7cb479a9ffa5d0a4c73b468 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:35.988264 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:35.990149 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:36.007304 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a5483-47fc-46a0-8db9-641c8b333218 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:36.012627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:36.014155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:36.014974 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:36.087847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df301cd-93e8-404d-8f5c-4e6f73212e68 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:36.092677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:36.093326 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.093604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.093876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc admin admin] PATH_INFO: `/v3/projects/fe5be6ce389040fe827f6c3734b551eb` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.095006 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5be6ce389040fe827f6c3734b551eb)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:36.096971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:36.106059 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2951a05-9948-4ded-bc3d-1e8ed5602a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:36.107022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2951a05-9948-4ded-bc3d-1e8ed5602a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.107431 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2951a05-9948-4ded-bc3d-1e8ed5602a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.107800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2951a05-9948-4ded-bc3d-1e8ed5602a15 admin admin] PATH_INFO: `/v3/users/71379027edf14a889da6e50978a8a41c` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.109213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2951a05-9948-4ded-bc3d-1e8ed5602a15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71379027edf14a889da6e50978a8a41c)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:36.111466 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2951a05-9948-4ded-bc3d-1e8ed5602a15 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:36.167025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5be6ce389040fe827f6c3734b551eb', 'request_id': 'req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.169036 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c2951a05-9948-4ded-bc3d-1e8ed5602a15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71379027edf14a889da6e50978a8a41c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.169953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:36.170704 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c2951a05-9948-4ded-bc3d-1e8ed5602a15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71379027edf14a889da6e50978a8a41c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.171273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c2951a05-9948-4ded-bc3d-1e8ed5602a15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71379027edf14a889da6e50978a8a41c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.177576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5be6ce389040fe827f6c3734b551eb', 'request_id': 'req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.186512 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5be6ce389040fe827f6c3734b551eb', 'request_id': 'req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.192361 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c2951a05-9948-4ded-bc3d-1e8ed5602a15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71379027edf14a889da6e50978a8a41c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.202271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c2951a05-9948-4ded-bc3d-1e8ed5602a15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71379027edf14a889da6e50978a8a41c'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.202481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc admin admin] The token cache is being invalidate because project fe5be6ce389040fe827f6c3734b551eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:36.202845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.203697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e0844e43-2ad0-4d8b-adfe-9a3f638232dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.205417 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1322/2630] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:36 2022] DELETE /identity/v3/projects/fe5be6ce389040fe827f6c3734b551eb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:36.210474 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b73370-c37f-4153-b359-cb2c57731491 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.210788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b73370-c37f-4153-b359-cb2c57731491 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.211065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b73370-c37f-4153-b359-cb2c57731491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.223180 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1309/2631] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:35 2022] DELETE /identity/v3/users/71379027edf14a889da6e50978a8a41c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:36.234927 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-66b73370-c37f-4153-b359-cb2c57731491 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:36.236394 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-66b73370-c37f-4153-b359-cb2c57731491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:36.322702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1323/2632] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:36.330481 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0c90e-c6be-4b07-9fa2-764383295342 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.331130 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0c90e-c6be-4b07-9fa2-764383295342 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.331460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0c90e-c6be-4b07-9fa2-764383295342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.357420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-07a0c90e-c6be-4b07-9fa2-764383295342 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:36.358667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-07a0c90e-c6be-4b07-9fa2-764383295342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:36.374097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1310/2633] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:36.378209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b73370-c37f-4153-b359-cb2c57731491 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:36.389882 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b73370-c37f-4153-b359-cb2c57731491 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:36.395511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797aecb1-ca62-44fd-bc48-f57f2c7a1748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:36.397074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aecb1-ca62-44fd-bc48-f57f2c7a1748 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.397074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aecb1-ca62-44fd-bc48-f57f2c7a1748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.397074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797aecb1-ca62-44fd-bc48-f57f2c7a1748 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.397904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797aecb1-ca62-44fd-bc48-f57f2c7a1748 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:36.398354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797aecb1-ca62-44fd-bc48-f57f2c7a1748 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:36.400640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797aecb1-ca62-44fd-bc48-f57f2c7a1748 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:36.420511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1324/2634] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:36.425225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0c90e-c6be-4b07-9fa2-764383295342 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:36.436157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0c90e-c6be-4b07-9fa2-764383295342 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:36.441940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796ae5d-10a6-4912-bec3-13d330ec9bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:36.442931 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796ae5d-10a6-4912-bec3-13d330ec9bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.443579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796ae5d-10a6-4912-bec3-13d330ec9bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.444084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796ae5d-10a6-4912-bec3-13d330ec9bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.444963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796ae5d-10a6-4912-bec3-13d330ec9bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:36.447598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796ae5d-10a6-4912-bec3-13d330ec9bc0 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:36.485814 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1311/2635] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:36 2022] POST /identity/v3/projects => generated 391 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:36.490026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797aecb1-ca62-44fd-bc48-f57f2c7a1748 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:36.501327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797aecb1-ca62-44fd-bc48-f57f2c7a1748 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:36.557996 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796ae5d-10a6-4912-bec3-13d330ec9bc0 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:36.560178 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2796ae5d-10a6-4912-bec3-13d330ec9bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:36.574943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0a885-7913-49ff-8de6-ce1ac3bc7cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:36.575825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0a885-7913-49ff-8de6-ce1ac3bc7cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.576251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0a885-7913-49ff-8de6-ce1ac3bc7cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.576760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0a885-7913-49ff-8de6-ce1ac3bc7cfa admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.577848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0a885-7913-49ff-8de6-ce1ac3bc7cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:36.594517 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0a885-7913-49ff-8de6-ce1ac3bc7cfa admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:36.650712 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:36.654727 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796ae5d-10a6-4912-bec3-13d330ec9bc0 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:36.660811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71ad46-f2c6-427c-a018-63c090fc5085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:36.661585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71ad46-f2c6-427c-a018-63c090fc5085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.662189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71ad46-f2c6-427c-a018-63c090fc5085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.662710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71ad46-f2c6-427c-a018-63c090fc5085 admin admin] PATH_INFO: `/v3/projects/591bdd6cf5f4479d9a00e43822ef3255` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.663257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1325/2636] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:36 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:36.667115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71ad46-f2c6-427c-a018-63c090fc5085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=591bdd6cf5f4479d9a00e43822ef3255)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:36.667493 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71ad46-f2c6-427c-a018-63c090fc5085 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:36.669318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0a885-7913-49ff-8de6-ce1ac3bc7cfa admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:36.683211 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0a885-7913-49ff-8de6-ce1ac3bc7cfa admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:36.696115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf797f6-7ce1-4857-8dc3-7f74179c41c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:36.697032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf797f6-7ce1-4857-8dc3-7f74179c41c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.697483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf797f6-7ce1-4857-8dc3-7f74179c41c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.697909 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf797f6-7ce1-4857-8dc3-7f74179c41c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.699324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf797f6-7ce1-4857-8dc3-7f74179c41c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:36.701278 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf797f6-7ce1-4857-8dc3-7f74179c41c2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:36.716481 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1326/2637] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:36.720941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf797f6-7ce1-4857-8dc3-7f74179c41c2 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:36.734747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf797f6-7ce1-4857-8dc3-7f74179c41c2 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:36.740051 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9f71ad46-f2c6-427c-a018-63c090fc5085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '591bdd6cf5f4479d9a00e43822ef3255', 'request_id': 'req-9f71ad46-f2c6-427c-a018-63c090fc5085'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.753689 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9f71ad46-f2c6-427c-a018-63c090fc5085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '591bdd6cf5f4479d9a00e43822ef3255', 'request_id': 'req-9f71ad46-f2c6-427c-a018-63c090fc5085'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.762338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9f71ad46-f2c6-427c-a018-63c090fc5085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '591bdd6cf5f4479d9a00e43822ef3255', 'request_id': 'req-9f71ad46-f2c6-427c-a018-63c090fc5085'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.774980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9f71ad46-f2c6-427c-a018-63c090fc5085 admin admin] The token cache is being invalidate because project 591bdd6cf5f4479d9a00e43822ef3255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:36.775320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9f71ad46-f2c6-427c-a018-63c090fc5085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.776191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-9f71ad46-f2c6-427c-a018-63c090fc5085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:36.778065 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1312/2638] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:36 2022] DELETE /identity/v3/projects/591bdd6cf5f4479d9a00e43822ef3255 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:36.783721 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8f9e9-fd62-4234-89e8-c00430b1dd61 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.784030 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8f9e9-fd62-4234-89e8-c00430b1dd61 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.784311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8f9e9-fd62-4234-89e8-c00430b1dd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.802029 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:36.807629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a9b8f9e9-fd62-4234-89e8-c00430b1dd61 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:36.809134 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a9b8f9e9-fd62-4234-89e8-c00430b1dd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:36.820992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a7f47-0362-4640-a4b3-acdda4924cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:36.821968 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a7f47-0362-4640-a4b3-acdda4924cbe admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.822469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a7f47-0362-4640-a4b3-acdda4924cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.823023 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a7f47-0362-4640-a4b3-acdda4924cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.824161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a7f47-0362-4640-a4b3-acdda4924cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:36.826774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a7f47-0362-4640-a4b3-acdda4924cbe admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:36.847690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1327/2639] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:36.849120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a7f47-0362-4640-a4b3-acdda4924cbe admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:36.851896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-018a7f47-0362-4640-a4b3-acdda4924cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:36.870906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a7f47-0362-4640-a4b3-acdda4924cbe admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:36.876804 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f54295-c63b-46ba-888d-e56d552ad9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:36.877642 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f54295-c63b-46ba-888d-e56d552ad9fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.877920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f54295-c63b-46ba-888d-e56d552ad9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.878634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f54295-c63b-46ba-888d-e56d552ad9fb admin admin] PATH_INFO: `/v3/projects/5ba8b7ea974943388411bb5c78cf6341/users/763b51eea7d64d3f9d75c6d9c6324127/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.902318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f54295-c63b-46ba-888d-e56d552ad9fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba8b7ea974943388411bb5c78cf6341, user_id=763b51eea7d64d3f9d75c6d9c6324127, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:36.906026 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f54295-c63b-46ba-888d-e56d552ad9fb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:36.913558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1313/2640] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:36.922588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e4791-04bc-4afa-9d2e-626ef705bc0f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.922873 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e4791-04bc-4afa-9d2e-626ef705bc0f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.923137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e4791-04bc-4afa-9d2e-626ef705bc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.925002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1328/2641] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:36 2022] PUT /identity/v3/projects/5ba8b7ea974943388411bb5c78cf6341/users/763b51eea7d64d3f9d75c6d9c6324127/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:36.937644 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76cc54-0e54-4126-a5d8-28c2757f7761 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.938240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76cc54-0e54-4126-a5d8-28c2757f7761 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.938690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76cc54-0e54-4126-a5d8-28c2757f7761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.958993 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-771e4791-04bc-4afa-9d2e-626ef705bc0f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:36.960579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-771e4791-04bc-4afa-9d2e-626ef705bc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:36.972467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-2e76cc54-0e54-4126-a5d8-28c2757f7761 None None] MFA Rules not processed for user `763b51eea7d64d3f9d75c6d9c6324127`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:36.978059 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2e76cc54-0e54-4126-a5d8-28c2757f7761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:37.041122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1314/2642] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:37.046476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e4791-04bc-4afa-9d2e-626ef705bc0f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.062015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e4791-04bc-4afa-9d2e-626ef705bc0f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.073943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1dd2b7-c948-485a-b034-114243fdd7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.075089 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1dd2b7-c948-485a-b034-114243fdd7cc admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.075971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1dd2b7-c948-485a-b034-114243fdd7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.077033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1dd2b7-c948-485a-b034-114243fdd7cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.078064 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1dd2b7-c948-485a-b034-114243fdd7cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:37.078628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1dd2b7-c948-485a-b034-114243fdd7cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.081741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1dd2b7-c948-485a-b034-114243fdd7cc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.083824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1329/2643] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:37.084845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76cc54-0e54-4126-a5d8-28c2757f7761 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.097206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76cc54-0e54-4126-a5d8-28c2757f7761 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.102587 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82676806-c216-4d70-9817-3cdfa69bf44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.103419 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82676806-c216-4d70-9817-3cdfa69bf44d admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.103803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82676806-c216-4d70-9817-3cdfa69bf44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.104135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82676806-c216-4d70-9817-3cdfa69bf44d admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.104957 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82676806-c216-4d70-9817-3cdfa69bf44d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.107175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82676806-c216-4d70-9817-3cdfa69bf44d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.109152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1315/2644] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:40:37.113572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1dd2b7-c948-485a-b034-114243fdd7cc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.128979 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1dd2b7-c948-485a-b034-114243fdd7cc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.137388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6f9d9-da79-48cc-853c-27f2553c525c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.138498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6f9d9-da79-48cc-853c-27f2553c525c admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.139982 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6f9d9-da79-48cc-853c-27f2553c525c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.140451 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6f9d9-da79-48cc-853c-27f2553c525c admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.141318 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6f9d9-da79-48cc-853c-27f2553c525c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.143656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6f9d9-da79-48cc-853c-27f2553c525c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.148487 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1330/2645] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:37 2022] POST /identity/v3/projects => generated 393 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:37.152996 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82676806-c216-4d70-9817-3cdfa69bf44d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.162487 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:37.184788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1316/2646] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:37 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:37.190675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6f9d9-da79-48cc-853c-27f2553c525c admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.234037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82676806-c216-4d70-9817-3cdfa69bf44d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.274467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:37.295938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6f9d9-da79-48cc-853c-27f2553c525c admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.302578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad306f51-3206-4987-b836-e753375cd266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.303103 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad306f51-3206-4987-b836-e753375cd266 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.303407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad306f51-3206-4987-b836-e753375cd266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.303906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad306f51-3206-4987-b836-e753375cd266 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.304667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad306f51-3206-4987-b836-e753375cd266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.307718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad306f51-3206-4987-b836-e753375cd266 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.309256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb33c54-cb46-4a3e-aced-ffe5aca70640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.309961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb33c54-cb46-4a3e-aced-ffe5aca70640 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.310306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb33c54-cb46-4a3e-aced-ffe5aca70640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.310645 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb33c54-cb46-4a3e-aced-ffe5aca70640 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.311342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb33c54-cb46-4a3e-aced-ffe5aca70640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.313379 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb33c54-cb46-4a3e-aced-ffe5aca70640 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.342772 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1317/2647] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:37 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:37.346909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad306f51-3206-4987-b836-e753375cd266 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.351591 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1331/2648] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:37 2022] POST /identity/v3/users => generated 303 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:37.353179 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb33c54-cb46-4a3e-aced-ffe5aca70640 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.361409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad306f51-3206-4987-b836-e753375cd266 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.364924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb33c54-cb46-4a3e-aced-ffe5aca70640 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.366424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14fe18-dd36-4f86-95a6-46acb705669e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.367035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14fe18-dd36-4f86-95a6-46acb705669e admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.367314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14fe18-dd36-4f86-95a6-46acb705669e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.367589 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14fe18-dd36-4f86-95a6-46acb705669e admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.368311 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14fe18-dd36-4f86-95a6-46acb705669e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.370209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14fe18-dd36-4f86-95a6-46acb705669e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.370742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232788a9-b411-44ec-ba4a-c12137f69e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.371649 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232788a9-b411-44ec-ba4a-c12137f69e9c admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.371935 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232788a9-b411-44ec-ba4a-c12137f69e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.372208 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232788a9-b411-44ec-ba4a-c12137f69e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.373131 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232788a9-b411-44ec-ba4a-c12137f69e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.374514 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232788a9-b411-44ec-ba4a-c12137f69e9c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.389603 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1318/2649] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:37.392992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1332/2650] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:37.392992 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232788a9-b411-44ec-ba4a-c12137f69e9c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.394324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14fe18-dd36-4f86-95a6-46acb705669e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.405813 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232788a9-b411-44ec-ba4a-c12137f69e9c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.409396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14fe18-dd36-4f86-95a6-46acb705669e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.411088 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a1c51-9784-4349-a611-57fb8bf2145a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.411827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a1c51-9784-4349-a611-57fb8bf2145a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.412240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a1c51-9784-4349-a611-57fb8bf2145a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.412547 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a1c51-9784-4349-a611-57fb8bf2145a admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.413380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a1c51-9784-4349-a611-57fb8bf2145a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.417663 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ca044-d4d5-4528-bc99-65dee53202a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.417837 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a1c51-9784-4349-a611-57fb8bf2145a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.418456 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ca044-d4d5-4528-bc99-65dee53202a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.418734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ca044-d4d5-4528-bc99-65dee53202a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.419410 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ca044-d4d5-4528-bc99-65dee53202a6 admin admin] PATH_INFO: `/v3/projects/29165ce0ffe74552b779ae3176d283ed/users/fa9d890a821a4dc0ae1253fdbfc25f5a/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.428043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1333/2651] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:37.433480 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a1c51-9784-4349-a611-57fb8bf2145a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.441579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ca044-d4d5-4528-bc99-65dee53202a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29165ce0ffe74552b779ae3176d283ed, user_id=fa9d890a821a4dc0ae1253fdbfc25f5a, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.444426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ca044-d4d5-4528-bc99-65dee53202a6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.446027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a1c51-9784-4349-a611-57fb8bf2145a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.452572 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a27759-5e04-4a60-8fcd-ddd92e90991e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.453576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a27759-5e04-4a60-8fcd-ddd92e90991e admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.454476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a27759-5e04-4a60-8fcd-ddd92e90991e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.454476 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a27759-5e04-4a60-8fcd-ddd92e90991e admin admin] PATH_INFO: `/v3/projects/f91c335e6207472087f03cb03e05db5b/users/53c33f8a32f04e60af189e6720ae0db2/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.469560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1319/2652] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:37 2022] PUT /identity/v3/projects/29165ce0ffe74552b779ae3176d283ed/users/fa9d890a821a4dc0ae1253fdbfc25f5a/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:37.473893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ca044-d4d5-4528-bc99-65dee53202a6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.484910 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a27759-5e04-4a60-8fcd-ddd92e90991e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f91c335e6207472087f03cb03e05db5b, user_id=53c33f8a32f04e60af189e6720ae0db2, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.488077 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a27759-5e04-4a60-8fcd-ddd92e90991e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.504217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1334/2653] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:37 2022] PUT /identity/v3/projects/f91c335e6207472087f03cb03e05db5b/users/53c33f8a32f04e60af189e6720ae0db2/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:37.515368 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00e6d9-bb31-435e-a95a-b063646b0722 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.515627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00e6d9-bb31-435e-a95a-b063646b0722 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.515892 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00e6d9-bb31-435e-a95a-b063646b0722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.554455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ca044-d4d5-4528-bc99-65dee53202a6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.559414 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7c00e6d9-bb31-435e-a95a-b063646b0722 None None] MFA Rules not processed for user `53c33f8a32f04e60af189e6720ae0db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:37.560808 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7c00e6d9-bb31-435e-a95a-b063646b0722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:37.613566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:37.623811 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750af22e-3387-41b3-bf06-eb02a2312e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.624513 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750af22e-3387-41b3-bf06-eb02a2312e6f admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.624820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750af22e-3387-41b3-bf06-eb02a2312e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.625086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750af22e-3387-41b3-bf06-eb02a2312e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.625784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750af22e-3387-41b3-bf06-eb02a2312e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.627459 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750af22e-3387-41b3-bf06-eb02a2312e6f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.635132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1335/2654] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:37.642327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1320/2655] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:37.646635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00e6d9-bb31-435e-a95a-b063646b0722 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.656588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750af22e-3387-41b3-bf06-eb02a2312e6f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.661899 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00e6d9-bb31-435e-a95a-b063646b0722 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.663521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-750af22e-3387-41b3-bf06-eb02a2312e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:37.667641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789e9c2-b729-480e-9d8b-c666d606bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.668666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789e9c2-b729-480e-9d8b-c666d606bd41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.668787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789e9c2-b729-480e-9d8b-c666d606bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.669018 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789e9c2-b729-480e-9d8b-c666d606bd41 admin admin] PATH_INFO: `/v3/projects/29165ce0ffe74552b779ae3176d283ed/users/fa9d890a821a4dc0ae1253fdbfc25f5a/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.670961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789e9c2-b729-480e-9d8b-c666d606bd41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29165ce0ffe74552b779ae3176d283ed, user_id=fa9d890a821a4dc0ae1253fdbfc25f5a, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.673760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789e9c2-b729-480e-9d8b-c666d606bd41 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.690135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1336/2656] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:37 2022] PUT /identity/v3/projects/29165ce0ffe74552b779ae3176d283ed/users/fa9d890a821a4dc0ae1253fdbfc25f5a/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:37.700787 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f6a79-a193-40b6-b724-431471702982 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.701102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f6a79-a193-40b6-b724-431471702982 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.701337 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f6a79-a193-40b6-b724-431471702982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.732995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-072f6a79-a193-40b6-b724-431471702982 None None] MFA Rules not processed for user `fa9d890a821a4dc0ae1253fdbfc25f5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:37.734234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-072f6a79-a193-40b6-b724-431471702982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:37.745127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750af22e-3387-41b3-bf06-eb02a2312e6f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.787093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:37.832397 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06457b45-9cf1-4356-85fe-7477a3597a5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.832634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1337/2657] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:37 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:37.833548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06457b45-9cf1-4356-85fe-7477a3597a5a service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.833895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06457b45-9cf1-4356-85fe-7477a3597a5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.834235 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06457b45-9cf1-4356-85fe-7477a3597a5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.846126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f6a79-a193-40b6-b724-431471702982 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:37.847566 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-072f6a79-a193-40b6-b724-431471702982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:37.866139 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f6a79-a193-40b6-b724-431471702982 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:37.881598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f829294-8e59-4dcd-8e57-3acb611b2f26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:37.883124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f829294-8e59-4dcd-8e57-3acb611b2f26 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:37.883655 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f829294-8e59-4dcd-8e57-3acb611b2f26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:37.884238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f829294-8e59-4dcd-8e57-3acb611b2f26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:37.902267 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f829294-8e59-4dcd-8e57-3acb611b2f26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.904110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06457b45-9cf1-4356-85fe-7477a3597a5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:37.904247 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f829294-8e59-4dcd-8e57-3acb611b2f26 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.905779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06457b45-9cf1-4356-85fe-7477a3597a5a service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:37.926864 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1338/2658] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:37 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:40:37.928839 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1321/2659] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:38.023926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f829294-8e59-4dcd-8e57-3acb611b2f26 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:38.033620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f829294-8e59-4dcd-8e57-3acb611b2f26 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:38.037674 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c07cd-b099-4c16-a206-f4223dcd0d90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:38.038280 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c07cd-b099-4c16-a206-f4223dcd0d90 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:38.038604 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c07cd-b099-4c16-a206-f4223dcd0d90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:38.038836 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c07cd-b099-4c16-a206-f4223dcd0d90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:38.147446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c07cd-b099-4c16-a206-f4223dcd0d90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:38.153383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c07cd-b099-4c16-a206-f4223dcd0d90 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:38.170342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1339/2660] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:38.389636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06457b45-9cf1-4356-85fe-7477a3597a5a service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:38.391348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-06457b45-9cf1-4356-85fe-7477a3597a5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:38.479799 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06457b45-9cf1-4356-85fe-7477a3597a5a service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:38.484745 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735b83c-e874-45ad-9f60-cf3f345b987d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:38.485381 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735b83c-e874-45ad-9f60-cf3f345b987d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:38.485684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735b83c-e874-45ad-9f60-cf3f345b987d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:38.485926 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735b83c-e874-45ad-9f60-cf3f345b987d admin admin] PATH_INFO: `/v3/users/fd39206c155f4e1cb9f7f90fb5a10556` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:38.487206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735b83c-e874-45ad-9f60-cf3f345b987d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd39206c155f4e1cb9f7f90fb5a10556)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:38.489387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735b83c-e874-45ad-9f60-cf3f345b987d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:38.492600 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:38.550509 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e735b83c-e874-45ad-9f60-cf3f345b987d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd39206c155f4e1cb9f7f90fb5a10556'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:38.551141 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e735b83c-e874-45ad-9f60-cf3f345b987d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd39206c155f4e1cb9f7f90fb5a10556'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:38.551779 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e735b83c-e874-45ad-9f60-cf3f345b987d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd39206c155f4e1cb9f7f90fb5a10556'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:38.566595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e735b83c-e874-45ad-9f60-cf3f345b987d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd39206c155f4e1cb9f7f90fb5a10556'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:38.577651 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-e735b83c-e874-45ad-9f60-cf3f345b987d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd39206c155f4e1cb9f7f90fb5a10556'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:38.597190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1322/2661] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:38 2022] DELETE /identity/v3/users/fd39206c155f4e1cb9f7f90fb5a10556 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:38.780927 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c07cd-b099-4c16-a206-f4223dcd0d90 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:38.792968 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c07cd-b099-4c16-a206-f4223dcd0d90 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:38.856755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d97373-295a-43c3-989f-c0f55c7bc7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:38.857402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d97373-295a-43c3-989f-c0f55c7bc7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:38.857683 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d97373-295a-43c3-989f-c0f55c7bc7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:38.857950 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d97373-295a-43c3-989f-c0f55c7bc7e6 admin admin] PATH_INFO: `/v3/projects/9095a2f3c7d946a2b017115364dc6753` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:38.859067 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d97373-295a-43c3-989f-c0f55c7bc7e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9095a2f3c7d946a2b017115364dc6753)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:38.861089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d97373-295a-43c3-989f-c0f55c7bc7e6 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:38.913209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06d97373-295a-43c3-989f-c0f55c7bc7e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9095a2f3c7d946a2b017115364dc6753', 'request_id': 'req-06d97373-295a-43c3-989f-c0f55c7bc7e6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:38.921955 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06d97373-295a-43c3-989f-c0f55c7bc7e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9095a2f3c7d946a2b017115364dc6753', 'request_id': 'req-06d97373-295a-43c3-989f-c0f55c7bc7e6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:38.928470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06d97373-295a-43c3-989f-c0f55c7bc7e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9095a2f3c7d946a2b017115364dc6753', 'request_id': 'req-06d97373-295a-43c3-989f-c0f55c7bc7e6'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:38.932634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:38.941400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06d97373-295a-43c3-989f-c0f55c7bc7e6 admin admin] The token cache is being invalidate because project 9095a2f3c7d946a2b017115364dc6753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:38.941871 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06d97373-295a-43c3-989f-c0f55c7bc7e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:38.942852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-06d97373-295a-43c3-989f-c0f55c7bc7e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:38.944392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1340/2662] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:38 2022] DELETE /identity/v3/projects/9095a2f3c7d946a2b017115364dc6753 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:38.949863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a79d80-2659-4263-8c81-e83944f10bea None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:38.950133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a79d80-2659-4263-8c81-e83944f10bea None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:38.950404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a79d80-2659-4263-8c81-e83944f10bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:38.970129 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-06a79d80-2659-4263-8c81-e83944f10bea None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:38.971325 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-06a79d80-2659-4263-8c81-e83944f10bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:39.032684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1323/2663] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:39.045652 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b4021-4c9f-45db-8c84-b8a84b506b32 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:39.046015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b4021-4c9f-45db-8c84-b8a84b506b32 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:39.046372 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b4021-4c9f-45db-8c84-b8a84b506b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:39.070539 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-de1b4021-4c9f-45db-8c84-b8a84b506b32 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:39.071775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-de1b4021-4c9f-45db-8c84-b8a84b506b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:39.082661 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1341/2664] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:39.090413 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a79d80-2659-4263-8c81-e83944f10bea None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:39.108937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a79d80-2659-4263-8c81-e83944f10bea None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:39.118709 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a66824-043b-4259-b66e-d58e7bbe5afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:39.119318 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a66824-043b-4259-b66e-d58e7bbe5afa admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:39.119588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a66824-043b-4259-b66e-d58e7bbe5afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:39.119861 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a66824-043b-4259-b66e-d58e7bbe5afa admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:39.120512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a66824-043b-4259-b66e-d58e7bbe5afa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:39.120860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a66824-043b-4259-b66e-d58e7bbe5afa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:39.122491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a66824-043b-4259-b66e-d58e7bbe5afa admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:39.136016 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1324/2665] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:39.140798 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b4021-4c9f-45db-8c84-b8a84b506b32 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:39.151169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b4021-4c9f-45db-8c84-b8a84b506b32 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:39.156515 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0662e3a-278a-41d8-b748-b2438dc73658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:39.157146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0662e3a-278a-41d8-b748-b2438dc73658 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:39.157422 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0662e3a-278a-41d8-b748-b2438dc73658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:39.157686 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0662e3a-278a-41d8-b748-b2438dc73658 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:39.158321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0662e3a-278a-41d8-b748-b2438dc73658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:39.160239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0662e3a-278a-41d8-b748-b2438dc73658 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:39.192896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1342/2666] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:39 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:39.198837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a66824-043b-4259-b66e-d58e7bbe5afa admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:39.209275 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a66824-043b-4259-b66e-d58e7bbe5afa admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:39.268107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015ff9c-f3d4-4ba0-bfbd-c7b57eab762d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:39.268734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015ff9c-f3d4-4ba0-bfbd-c7b57eab762d admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:39.269015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015ff9c-f3d4-4ba0-bfbd-c7b57eab762d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:39.269288 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015ff9c-f3d4-4ba0-bfbd-c7b57eab762d admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:39.269919 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015ff9c-f3d4-4ba0-bfbd-c7b57eab762d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:39.271859 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015ff9c-f3d4-4ba0-bfbd-c7b57eab762d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:39.292444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:39.299290 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1325/2667] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:39 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:39.303599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0662e3a-278a-41d8-b748-b2438dc73658 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:39.314403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0662e3a-278a-41d8-b748-b2438dc73658 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:39.319132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c5444-5d73-4355-a0b1-b03b2380dcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:39.319729 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c5444-5d73-4355-a0b1-b03b2380dcfb admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:39.320008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c5444-5d73-4355-a0b1-b03b2380dcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:39.320277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c5444-5d73-4355-a0b1-b03b2380dcfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:39.320956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c5444-5d73-4355-a0b1-b03b2380dcfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:39.322901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c5444-5d73-4355-a0b1-b03b2380dcfb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:39.333017 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1343/2668] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:39.337282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015ff9c-f3d4-4ba0-bfbd-c7b57eab762d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:39.347967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015ff9c-f3d4-4ba0-bfbd-c7b57eab762d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:39.353077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457a44f-f18f-4b6f-a164-fcad80caab5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:39.353681 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457a44f-f18f-4b6f-a164-fcad80caab5c admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:39.353977 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457a44f-f18f-4b6f-a164-fcad80caab5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:39.354247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457a44f-f18f-4b6f-a164-fcad80caab5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:39.354869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457a44f-f18f-4b6f-a164-fcad80caab5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:39.356512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457a44f-f18f-4b6f-a164-fcad80caab5c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:39.367867 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1326/2669] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:39.372230 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c5444-5d73-4355-a0b1-b03b2380dcfb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:39.383071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c5444-5d73-4355-a0b1-b03b2380dcfb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:39.388834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0886b5-f313-40e0-b40b-bd87b42b8a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:39.389507 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0886b5-f313-40e0-b40b-bd87b42b8a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:39.389783 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0886b5-f313-40e0-b40b-bd87b42b8a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:39.390035 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0886b5-f313-40e0-b40b-bd87b42b8a57 admin admin] PATH_INFO: `/v3/projects/7a0cc4dc0928432aaba22a58e9401393/users/86f7e2ec66c048b8a3952a9328ccd237/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:39.406393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0886b5-f313-40e0-b40b-bd87b42b8a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a0cc4dc0928432aaba22a58e9401393, user_id=86f7e2ec66c048b8a3952a9328ccd237, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:39.409130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0886b5-f313-40e0-b40b-bd87b42b8a57 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:39.425072 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1344/2670] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:39 2022] PUT /identity/v3/projects/7a0cc4dc0928432aaba22a58e9401393/users/86f7e2ec66c048b8a3952a9328ccd237/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:39.447960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f304f-5963-4e08-8369-3c3b0682fcc8 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:39.447960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f304f-5963-4e08-8369-3c3b0682fcc8 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:39.447960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f304f-5963-4e08-8369-3c3b0682fcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:39.486269 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-eb5f304f-5963-4e08-8369-3c3b0682fcc8 None None] MFA Rules not processed for user `86f7e2ec66c048b8a3952a9328ccd237`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:39.487564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eb5f304f-5963-4e08-8369-3c3b0682fcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:39.565352 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1327/2671] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:39.572239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0886b5-f313-40e0-b40b-bd87b42b8a57 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:39.657741 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0886b5-f313-40e0-b40b-bd87b42b8a57 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:39.660359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cc3e0-c36c-4e4a-bb2c-d46ffd9ecd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:39.661392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cc3e0-c36c-4e4a-bb2c-d46ffd9ecd76 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:39.661676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cc3e0-c36c-4e4a-bb2c-d46ffd9ecd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:39.661954 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cc3e0-c36c-4e4a-bb2c-d46ffd9ecd76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:39.662733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623cc3e0-c36c-4e4a-bb2c-d46ffd9ecd76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:39.665215 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623cc3e0-c36c-4e4a-bb2c-d46ffd9ecd76 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:39.700747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1345/2672] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:39 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:39.704557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f304f-5963-4e08-8369-3c3b0682fcc8 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:39.717328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f304f-5963-4e08-8369-3c3b0682fcc8 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:39.790053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95f09a-581c-44d9-b361-a378682764dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:39.790863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95f09a-581c-44d9-b361-a378682764dd admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:39.791165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95f09a-581c-44d9-b361-a378682764dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:39.791529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95f09a-581c-44d9-b361-a378682764dd admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:39.792868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95f09a-581c-44d9-b361-a378682764dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:39.795007 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95f09a-581c-44d9-b361-a378682764dd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:39.824686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1328/2673] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:39 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:39.829164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cc3e0-c36c-4e4a-bb2c-d46ffd9ecd76 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:39.844944 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cc3e0-c36c-4e4a-bb2c-d46ffd9ecd76 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:39.851040 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bd4f0-d9bf-4b7d-a172-fabff45e8f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:39.851834 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bd4f0-d9bf-4b7d-a172-fabff45e8f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:39.852321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bd4f0-d9bf-4b7d-a172-fabff45e8f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:39.852773 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bd4f0-d9bf-4b7d-a172-fabff45e8f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:39.858008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bd4f0-d9bf-4b7d-a172-fabff45e8f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:39.858008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bd4f0-d9bf-4b7d-a172-fabff45e8f28 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:39.867582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1346/2674] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:39.872160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95f09a-581c-44d9-b361-a378682764dd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:39.882500 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95f09a-581c-44d9-b361-a378682764dd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:39.887750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934822c-0280-41f6-9a53-fed0a4215aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:39.888525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934822c-0280-41f6-9a53-fed0a4215aae admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:39.888914 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934822c-0280-41f6-9a53-fed0a4215aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:39.889267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934822c-0280-41f6-9a53-fed0a4215aae admin admin] PATH_INFO: `/v3/projects/0d698e62c91e4837a1e624b93b16937d/users/66aa92edeec448aebeef72ae6987e8ad/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:39.906071 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934822c-0280-41f6-9a53-fed0a4215aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d698e62c91e4837a1e624b93b16937d, user_id=66aa92edeec448aebeef72ae6987e8ad, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:39.908789 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934822c-0280-41f6-9a53-fed0a4215aae admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:39.925040 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1329/2675] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:39 2022] PUT /identity/v3/projects/0d698e62c91e4837a1e624b93b16937d/users/66aa92edeec448aebeef72ae6987e8ad/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:39.929186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bd4f0-d9bf-4b7d-a172-fabff45e8f28 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:39.939637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bd4f0-d9bf-4b7d-a172-fabff45e8f28 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:40.018160 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52119770-682c-4815-a5af-13c49eeeda9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:40.018841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52119770-682c-4815-a5af-13c49eeeda9f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:40.019119 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52119770-682c-4815-a5af-13c49eeeda9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:40.019393 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52119770-682c-4815-a5af-13c49eeeda9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:40.020249 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52119770-682c-4815-a5af-13c49eeeda9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:40.022110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52119770-682c-4815-a5af-13c49eeeda9f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:40.029342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:40.035430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1347/2676] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:40.040365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934822c-0280-41f6-9a53-fed0a4215aae admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:40.053179 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934822c-0280-41f6-9a53-fed0a4215aae admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:40.058065 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1475e13-b268-4800-951a-22b21ea907dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:40.058729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1475e13-b268-4800-951a-22b21ea907dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:40.059005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1475e13-b268-4800-951a-22b21ea907dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:40.059277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1475e13-b268-4800-951a-22b21ea907dd admin admin] PATH_INFO: `/v3/projects/0d698e62c91e4837a1e624b93b16937d/users/66aa92edeec448aebeef72ae6987e8ad/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:40.061542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1475e13-b268-4800-951a-22b21ea907dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d698e62c91e4837a1e624b93b16937d, user_id=66aa92edeec448aebeef72ae6987e8ad, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:40.067535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1475e13-b268-4800-951a-22b21ea907dd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:40.081226 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1330/2677] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:40 2022] PUT /identity/v3/projects/0d698e62c91e4837a1e624b93b16937d/users/66aa92edeec448aebeef72ae6987e8ad/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:40.091982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc546f-fad9-421c-9004-68e0f4eff03a None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:40.092397 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc546f-fad9-421c-9004-68e0f4eff03a None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:40.092819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc546f-fad9-421c-9004-68e0f4eff03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:40.125259 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-6adc546f-fad9-421c-9004-68e0f4eff03a None None] MFA Rules not processed for user `66aa92edeec448aebeef72ae6987e8ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:40.126529 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-6adc546f-fad9-421c-9004-68e0f4eff03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:40.218361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1348/2678] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:40.230067 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1475e13-b268-4800-951a-22b21ea907dd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:40.231917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c1475e13-b268-4800-951a-22b21ea907dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:40.264495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:40.292743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1475e13-b268-4800-951a-22b21ea907dd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:40.296497 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825b5f0-b9b5-42f6-8ae8-ad80dfae0eb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:40.297127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825b5f0-b9b5-42f6-8ae8-ad80dfae0eb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:40.297401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825b5f0-b9b5-42f6-8ae8-ad80dfae0eb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:40.297669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825b5f0-b9b5-42f6-8ae8-ad80dfae0eb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:40.307284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825b5f0-b9b5-42f6-8ae8-ad80dfae0eb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:40.308759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825b5f0-b9b5-42f6-8ae8-ad80dfae0eb0 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:40.318707 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1331/2679] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:40.408724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc546f-fad9-421c-9004-68e0f4eff03a None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:40.419719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc546f-fad9-421c-9004-68e0f4eff03a None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:40.423915 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b9387-30cb-447a-aabf-94cb4633aba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:40.424602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b9387-30cb-447a-aabf-94cb4633aba9 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:40.424971 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b9387-30cb-447a-aabf-94cb4633aba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:40.425330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b9387-30cb-447a-aabf-94cb4633aba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:40.496246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b9387-30cb-447a-aabf-94cb4633aba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:40.497737 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b9387-30cb-447a-aabf-94cb4633aba9 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:40.507079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1349/2680] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:41.202145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825b5f0-b9b5-42f6-8ae8-ad80dfae0eb0 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:41.203763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7825b5f0-b9b5-42f6-8ae8-ad80dfae0eb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:41.274584 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825b5f0-b9b5-42f6-8ae8-ad80dfae0eb0 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:41.282615 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:41.283247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:41.283524 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:41.283795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] PATH_INFO: `/v3/users/53c33f8a32f04e60af189e6720ae0db2` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:41.285559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53c33f8a32f04e60af189e6720ae0db2)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:41.288269 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:41.349372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53c33f8a32f04e60af189e6720ae0db2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:41.350035 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53c33f8a32f04e60af189e6720ae0db2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:41.350647 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53c33f8a32f04e60af189e6720ae0db2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:41.362846 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53c33f8a32f04e60af189e6720ae0db2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:41.371726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53c33f8a32f04e60af189e6720ae0db2'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:41.384207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1332/2681] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:41 2022] DELETE /identity/v3/users/53c33f8a32f04e60af189e6720ae0db2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:41.388906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35547c74-3101-4627-a460-ea6323e3f6df None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:41.389376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35547c74-3101-4627-a460-ea6323e3f6df None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:41.389473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35547c74-3101-4627-a460-ea6323e3f6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:41.408305 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-35547c74-3101-4627-a460-ea6323e3f6df None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:41.409515 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-35547c74-3101-4627-a460-ea6323e3f6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:41.469678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1350/2682] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:41.479730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:41.481281 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:41.545759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888c6ce-66a2-41b7-9b59-258e1f005e93 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:41.549833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b11b08-e117-4fa7-80b1-e0f38ec290ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:41.550449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b11b08-e117-4fa7-80b1-e0f38ec290ef service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:41.550726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b11b08-e117-4fa7-80b1-e0f38ec290ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:41.550998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b11b08-e117-4fa7-80b1-e0f38ec290ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:41.558516 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:41.563496 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b11b08-e117-4fa7-80b1-e0f38ec290ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:41.565706 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b11b08-e117-4fa7-80b1-e0f38ec290ef service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:41.577717 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1333/2683] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:41.748503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35547c74-3101-4627-a460-ea6323e3f6df None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:41.750348 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-35547c74-3101-4627-a460-ea6323e3f6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:41.763149 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35547c74-3101-4627-a460-ea6323e3f6df None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:41.768461 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973d9b3-d84d-4ef8-b87f-b79297db066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:41.769334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973d9b3-d84d-4ef8-b87f-b79297db066b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:41.769788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973d9b3-d84d-4ef8-b87f-b79297db066b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:41.770352 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973d9b3-d84d-4ef8-b87f-b79297db066b admin admin] PATH_INFO: `/v3/users/763b51eea7d64d3f9d75c6d9c6324127` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:41.772193 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973d9b3-d84d-4ef8-b87f-b79297db066b admin admin] RBAC: Authorizing `identity:delete_user(user_id=763b51eea7d64d3f9d75c6d9c6324127)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:41.774672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973d9b3-d84d-4ef8-b87f-b79297db066b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:41.804827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:41.857253 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4973d9b3-d84d-4ef8-b87f-b79297db066b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '763b51eea7d64d3f9d75c6d9c6324127'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:41.857709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4973d9b3-d84d-4ef8-b87f-b79297db066b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '763b51eea7d64d3f9d75c6d9c6324127'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:41.858316 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4973d9b3-d84d-4ef8-b87f-b79297db066b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '763b51eea7d64d3f9d75c6d9c6324127'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:41.868054 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b11b08-e117-4fa7-80b1-e0f38ec290ef service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:41.870428 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-93b11b08-e117-4fa7-80b1-e0f38ec290ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:41.878637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4973d9b3-d84d-4ef8-b87f-b79297db066b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '763b51eea7d64d3f9d75c6d9c6324127'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:41.889294 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4973d9b3-d84d-4ef8-b87f-b79297db066b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '763b51eea7d64d3f9d75c6d9c6324127'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:41.901292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b11b08-e117-4fa7-80b1-e0f38ec290ef service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:41.906732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1351/2684] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:41 2022] DELETE /identity/v3/users/763b51eea7d64d3f9d75c6d9c6324127 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:41.910462 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d14883-67ed-4aca-b91d-d8dd57ea95b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:41.911222 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d14883-67ed-4aca-b91d-d8dd57ea95b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:41.911515 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f6039-74e5-4a25-88c8-50b9c678fd05 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:41.911623 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d14883-67ed-4aca-b91d-d8dd57ea95b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:41.911973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f6039-74e5-4a25-88c8-50b9c678fd05 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:41.912093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d14883-67ed-4aca-b91d-d8dd57ea95b9 admin admin] PATH_INFO: `/v3/projects/f91c335e6207472087f03cb03e05db5b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:41.912328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f6039-74e5-4a25-88c8-50b9c678fd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:41.913401 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d14883-67ed-4aca-b91d-d8dd57ea95b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f91c335e6207472087f03cb03e05db5b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:41.916581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d14883-67ed-4aca-b91d-d8dd57ea95b9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:41.934952 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-fa3f6039-74e5-4a25-88c8-50b9c678fd05 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:41.936491 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fa3f6039-74e5-4a25-88c8-50b9c678fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:41.985404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-63d14883-67ed-4aca-b91d-d8dd57ea95b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f91c335e6207472087f03cb03e05db5b', 'request_id': 'req-63d14883-67ed-4aca-b91d-d8dd57ea95b9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.003222 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-63d14883-67ed-4aca-b91d-d8dd57ea95b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f91c335e6207472087f03cb03e05db5b', 'request_id': 'req-63d14883-67ed-4aca-b91d-d8dd57ea95b9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.013747 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-63d14883-67ed-4aca-b91d-d8dd57ea95b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f91c335e6207472087f03cb03e05db5b', 'request_id': 'req-63d14883-67ed-4aca-b91d-d8dd57ea95b9'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.017939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1352/2685] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:42.027102 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f6039-74e5-4a25-88c8-50b9c678fd05 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.028872 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fa3f6039-74e5-4a25-88c8-50b9c678fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.048522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-63d14883-67ed-4aca-b91d-d8dd57ea95b9 admin admin] The token cache is being invalidate because project f91c335e6207472087f03cb03e05db5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:42.049270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-63d14883-67ed-4aca-b91d-d8dd57ea95b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.051213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-63d14883-67ed-4aca-b91d-d8dd57ea95b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.053068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1334/2686] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:41 2022] DELETE /identity/v3/projects/f91c335e6207472087f03cb03e05db5b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:42.058974 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f26868-524b-4ac4-a06b-d1d231b62b04 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.066018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f26868-524b-4ac4-a06b-d1d231b62b04 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.066018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f26868-524b-4ac4-a06b-d1d231b62b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.095594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-46f26868-524b-4ac4-a06b-d1d231b62b04 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:42.098665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-46f26868-524b-4ac4-a06b-d1d231b62b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.139936 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f6039-74e5-4a25-88c8-50b9c678fd05 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.170378 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1335/2687] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:42.178272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c386a-35e8-43c8-85fa-0a922a8e605d None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.178558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c386a-35e8-43c8-85fa-0a922a8e605d None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.178834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c386a-35e8-43c8-85fa-0a922a8e605d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.192889 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8e867-3068-4206-86b6-86d2b21e2262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.193991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8e867-3068-4206-86b6-86d2b21e2262 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.194385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8e867-3068-4206-86b6-86d2b21e2262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.194660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8e867-3068-4206-86b6-86d2b21e2262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.196437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fcc8e867-3068-4206-86b6-86d2b21e2262 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.200714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a80c386a-35e8-43c8-85fa-0a922a8e605d None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:42.201915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a80c386a-35e8-43c8-85fa-0a922a8e605d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.206506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:42.209319 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8e867-3068-4206-86b6-86d2b21e2262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.211281 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8e867-3068-4206-86b6-86d2b21e2262 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.211406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1336/2688] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:42.216539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c386a-35e8-43c8-85fa-0a922a8e605d None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.225890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c386a-35e8-43c8-85fa-0a922a8e605d None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.229014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1353/2689] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:40:42.232763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32f67d-ba38-4414-8c65-1a908ab59bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.232763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32f67d-ba38-4414-8c65-1a908ab59bed admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.232763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32f67d-ba38-4414-8c65-1a908ab59bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.233125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32f67d-ba38-4414-8c65-1a908ab59bed admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.233716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32f67d-ba38-4414-8c65-1a908ab59bed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:42.234022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32f67d-ba38-4414-8c65-1a908ab59bed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.235638 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32f67d-ba38-4414-8c65-1a908ab59bed admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.252372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1337/2690] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:40:42.253830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8e867-3068-4206-86b6-86d2b21e2262 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.264754 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8e867-3068-4206-86b6-86d2b21e2262 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.269615 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4578608-1b70-443b-85b4-facf222ab2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.270228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4578608-1b70-443b-85b4-facf222ab2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.270508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4578608-1b70-443b-85b4-facf222ab2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.270779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4578608-1b70-443b-85b4-facf222ab2e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.271408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4578608-1b70-443b-85b4-facf222ab2e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.273535 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4578608-1b70-443b-85b4-facf222ab2e5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.307995 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1354/2691] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:42 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:42.309768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32f67d-ba38-4414-8c65-1a908ab59bed admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.320318 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32f67d-ba38-4414-8c65-1a908ab59bed admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.387429 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:42.395320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b353f-e88c-43de-a4c5-55fc82aa42b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.396350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b353f-e88c-43de-a4c5-55fc82aa42b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.396670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b353f-e88c-43de-a4c5-55fc82aa42b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.396950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b353f-e88c-43de-a4c5-55fc82aa42b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.397637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b353f-e88c-43de-a4c5-55fc82aa42b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.399591 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b353f-e88c-43de-a4c5-55fc82aa42b9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.422305 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1338/2692] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:42 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:42.426636 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4578608-1b70-443b-85b4-facf222ab2e5 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.437223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4578608-1b70-443b-85b4-facf222ab2e5 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.442343 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a501728-9a87-477a-af80-2f70dd58500f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.443160 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a501728-9a87-477a-af80-2f70dd58500f admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.443496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a501728-9a87-477a-af80-2f70dd58500f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.443832 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a501728-9a87-477a-af80-2f70dd58500f admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.444554 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a501728-9a87-477a-af80-2f70dd58500f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.446341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a501728-9a87-477a-af80-2f70dd58500f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.457534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1355/2693] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:42.461624 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b353f-e88c-43de-a4c5-55fc82aa42b9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.473949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b353f-e88c-43de-a4c5-55fc82aa42b9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.479463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e05596-b8ca-4814-80b2-1c169a844f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.480605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e05596-b8ca-4814-80b2-1c169a844f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.480965 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e05596-b8ca-4814-80b2-1c169a844f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.481307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e05596-b8ca-4814-80b2-1c169a844f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.482055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e05596-b8ca-4814-80b2-1c169a844f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.483939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e05596-b8ca-4814-80b2-1c169a844f09 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.496098 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1339/2694] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:42.501009 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a501728-9a87-477a-af80-2f70dd58500f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.514209 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a501728-9a87-477a-af80-2f70dd58500f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.521745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6856c-b90a-4811-b171-2799a5580e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.522390 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6856c-b90a-4811-b171-2799a5580e9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.522733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e05596-b8ca-4814-80b2-1c169a844f09 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.522879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6856c-b90a-4811-b171-2799a5580e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.524244 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6856c-b90a-4811-b171-2799a5580e9c admin admin] PATH_INFO: `/v3/projects/bca754f41a1e4349a8268eba44d16d2c/users/69cd8325f60f4eb7a1394b0caf06582a/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.524582 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e7e05596-b8ca-4814-80b2-1c169a844f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.551044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6856c-b90a-4811-b171-2799a5580e9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca754f41a1e4349a8268eba44d16d2c, user_id=69cd8325f60f4eb7a1394b0caf06582a, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.553956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6856c-b90a-4811-b171-2799a5580e9c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.577437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1356/2695] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:42 2022] PUT /identity/v3/projects/bca754f41a1e4349a8268eba44d16d2c/users/69cd8325f60f4eb7a1394b0caf06582a/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:42.587861 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d9627-cded-48f3-a3c4-c136028e95be None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.588150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d9627-cded-48f3-a3c4-c136028e95be None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.589048 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d9627-cded-48f3-a3c4-c136028e95be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.614878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e05596-b8ca-4814-80b2-1c169a844f09 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.616743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-534d9627-cded-48f3-a3c4-c136028e95be None None] MFA Rules not processed for user `69cd8325f60f4eb7a1394b0caf06582a`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:42.618134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-534d9627-cded-48f3-a3c4-c136028e95be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.672337 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.673227 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.673507 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.673778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] PATH_INFO: `/v3/projects/5ba8b7ea974943388411bb5c78cf6341` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.674917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba8b7ea974943388411bb5c78cf6341)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.676854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.685386 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1357/2696] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:42.697876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d9627-cded-48f3-a3c4-c136028e95be None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.699320 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-534d9627-cded-48f3-a3c4-c136028e95be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.725106 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba8b7ea974943388411bb5c78cf6341', 'request_id': 'req-eafa09bb-accc-4747-bf2c-9a2fff110bc6'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.733320 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba8b7ea974943388411bb5c78cf6341', 'request_id': 'req-eafa09bb-accc-4747-bf2c-9a2fff110bc6'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.739895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba8b7ea974943388411bb5c78cf6341', 'request_id': 'req-eafa09bb-accc-4747-bf2c-9a2fff110bc6'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.751043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] The token cache is being invalidate because project 5ba8b7ea974943388411bb5c78cf6341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:42.751334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.752051 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.753697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1340/2697] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:42 2022] DELETE /identity/v3/projects/5ba8b7ea974943388411bb5c78cf6341 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:42.757900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.759321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.762827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d9627-cded-48f3-a3c4-c136028e95be None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.765800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:42.800300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:42.827652 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a872508-c609-40b2-a36d-f34093593b37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.828354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a872508-c609-40b2-a36d-f34093593b37 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.828635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a872508-c609-40b2-a36d-f34093593b37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.828953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a872508-c609-40b2-a36d-f34093593b37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.830022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa09bb-accc-4747-bf2c-9a2fff110bc6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.830782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-2a872508-c609-40b2-a36d-f34093593b37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.834850 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b6e6a-97b8-4969-9170-504c7e25e4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.835479 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b6e6a-97b8-4969-9170-504c7e25e4d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.835757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b6e6a-97b8-4969-9170-504c7e25e4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.836028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b6e6a-97b8-4969-9170-504c7e25e4d7 admin admin] PATH_INFO: `/v3/users/fa9d890a821a4dc0ae1253fdbfc25f5a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.837236 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b6e6a-97b8-4969-9170-504c7e25e4d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa9d890a821a4dc0ae1253fdbfc25f5a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.839348 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b6e6a-97b8-4969-9170-504c7e25e4d7 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.891242 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c94b6e6a-97b8-4969-9170-504c7e25e4d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa9d890a821a4dc0ae1253fdbfc25f5a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.891893 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c94b6e6a-97b8-4969-9170-504c7e25e4d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa9d890a821a4dc0ae1253fdbfc25f5a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.892529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c94b6e6a-97b8-4969-9170-504c7e25e4d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa9d890a821a4dc0ae1253fdbfc25f5a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.898078 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a872508-c609-40b2-a36d-f34093593b37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.899736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a872508-c609-40b2-a36d-f34093593b37 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.905963 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c94b6e6a-97b8-4969-9170-504c7e25e4d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa9d890a821a4dc0ae1253fdbfc25f5a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.910702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1358/2698] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:40:42.914782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-c94b6e6a-97b8-4969-9170-504c7e25e4d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa9d890a821a4dc0ae1253fdbfc25f5a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.929521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1341/2699] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:42 2022] DELETE /identity/v3/users/fa9d890a821a4dc0ae1253fdbfc25f5a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:43.225899 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a872508-c609-40b2-a36d-f34093593b37 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.241944 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a872508-c609-40b2-a36d-f34093593b37 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.329983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f5818-fa95-4018-9d5f-7dd272980411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.330789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f5818-fa95-4018-9d5f-7dd272980411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.331556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f5818-fa95-4018-9d5f-7dd272980411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.331556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f5818-fa95-4018-9d5f-7dd272980411 admin admin] PATH_INFO: `/v3/projects/29165ce0ffe74552b779ae3176d283ed` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.332813 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f5818-fa95-4018-9d5f-7dd272980411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29165ce0ffe74552b779ae3176d283ed)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.334803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f5818-fa95-4018-9d5f-7dd272980411 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.385797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-359f5818-fa95-4018-9d5f-7dd272980411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29165ce0ffe74552b779ae3176d283ed', 'request_id': 'req-359f5818-fa95-4018-9d5f-7dd272980411'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.393015 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-359f5818-fa95-4018-9d5f-7dd272980411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29165ce0ffe74552b779ae3176d283ed', 'request_id': 'req-359f5818-fa95-4018-9d5f-7dd272980411'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.399511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-359f5818-fa95-4018-9d5f-7dd272980411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29165ce0ffe74552b779ae3176d283ed', 'request_id': 'req-359f5818-fa95-4018-9d5f-7dd272980411'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.411698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-359f5818-fa95-4018-9d5f-7dd272980411 admin admin] The token cache is being invalidate because project 29165ce0ffe74552b779ae3176d283ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:43.412014 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-359f5818-fa95-4018-9d5f-7dd272980411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.412746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-359f5818-fa95-4018-9d5f-7dd272980411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.415252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1359/2700] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:43 2022] DELETE /identity/v3/projects/29165ce0ffe74552b779ae3176d283ed => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:43.419886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd614ac-809f-4fe9-9de4-27af89b6f999 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.420374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd614ac-809f-4fe9-9de4-27af89b6f999 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.420595 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd614ac-809f-4fe9-9de4-27af89b6f999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.485862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ddd614ac-809f-4fe9-9de4-27af89b6f999 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:43.487247 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ddd614ac-809f-4fe9-9de4-27af89b6f999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:43.550403 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:43.567691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1342/2701] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:43.577529 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ba22c-b96a-46d6-b600-6f1f4c486e34 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.578277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ba22c-b96a-46d6-b600-6f1f4c486e34 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.578277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ba22c-b96a-46d6-b600-6f1f4c486e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.607732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-c72ba22c-b96a-46d6-b600-6f1f4c486e34 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:43.608986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-c72ba22c-b96a-46d6-b600-6f1f4c486e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:43.617820 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1360/2702] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:43.622838 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd614ac-809f-4fe9-9de4-27af89b6f999 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.633357 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd614ac-809f-4fe9-9de4-27af89b6f999 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.639077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905f584-1ffb-4d62-93d4-d67e31ac432f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.639669 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905f584-1ffb-4d62-93d4-d67e31ac432f admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.639939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905f584-1ffb-4d62-93d4-d67e31ac432f admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.640199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905f584-1ffb-4d62-93d4-d67e31ac432f admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.640874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905f584-1ffb-4d62-93d4-d67e31ac432f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:43.641170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905f584-1ffb-4d62-93d4-d67e31ac432f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.642772 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905f584-1ffb-4d62-93d4-d67e31ac432f admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.654941 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1343/2703] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:43.661641 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ba22c-b96a-46d6-b600-6f1f4c486e34 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.675903 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ba22c-b96a-46d6-b600-6f1f4c486e34 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.683256 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d37661-4196-45fd-ba0b-9e281c73a830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.683969 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d37661-4196-45fd-ba0b-9e281c73a830 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.684251 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d37661-4196-45fd-ba0b-9e281c73a830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.684581 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d37661-4196-45fd-ba0b-9e281c73a830 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.685342 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d37661-4196-45fd-ba0b-9e281c73a830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.687375 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d37661-4196-45fd-ba0b-9e281c73a830 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.692227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:43.723226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1361/2704] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:43 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:43.727690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905f584-1ffb-4d62-93d4-d67e31ac432f admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.737890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905f584-1ffb-4d62-93d4-d67e31ac432f admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.799184 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535df715-fcde-44fd-bde3-194242a7a96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.799847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535df715-fcde-44fd-bde3-194242a7a96d admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.800147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535df715-fcde-44fd-bde3-194242a7a96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.800475 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535df715-fcde-44fd-bde3-194242a7a96d admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.801112 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535df715-fcde-44fd-bde3-194242a7a96d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.803124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535df715-fcde-44fd-bde3-194242a7a96d admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.826555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1344/2705] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:43 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:43.831425 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d37661-4196-45fd-ba0b-9e281c73a830 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.842579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d37661-4196-45fd-ba0b-9e281c73a830 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.847636 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0eac2-7019-4d1f-a037-d870ba32c958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.848227 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0eac2-7019-4d1f-a037-d870ba32c958 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.848503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0eac2-7019-4d1f-a037-d870ba32c958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.848800 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0eac2-7019-4d1f-a037-d870ba32c958 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.849584 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0eac2-7019-4d1f-a037-d870ba32c958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.851192 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0eac2-7019-4d1f-a037-d870ba32c958 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.862175 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1362/2706] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:43.866321 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535df715-fcde-44fd-bde3-194242a7a96d admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.878169 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535df715-fcde-44fd-bde3-194242a7a96d admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.885698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7451c5-b592-44a0-8ca9-cde3378eb8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.886283 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7451c5-b592-44a0-8ca9-cde3378eb8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.886555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7451c5-b592-44a0-8ca9-cde3378eb8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.886819 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7451c5-b592-44a0-8ca9-cde3378eb8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.887454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7451c5-b592-44a0-8ca9-cde3378eb8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.889117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7451c5-b592-44a0-8ca9-cde3378eb8a6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.899382 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1345/2707] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:43.904479 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0eac2-7019-4d1f-a037-d870ba32c958 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.915684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0eac2-7019-4d1f-a037-d870ba32c958 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.921303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef8bdc-01dd-48a4-b9b1-7e41a2fda704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.921956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef8bdc-01dd-48a4-b9b1-7e41a2fda704 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.922239 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef8bdc-01dd-48a4-b9b1-7e41a2fda704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.922512 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef8bdc-01dd-48a4-b9b1-7e41a2fda704 admin admin] PATH_INFO: `/v3/projects/df73feb9212c4398a6528d6c183b0d29/users/6c328eb0742a4a0f99ab6e9f491d5eb3/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.939036 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef8bdc-01dd-48a4-b9b1-7e41a2fda704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df73feb9212c4398a6528d6c183b0d29, user_id=6c328eb0742a4a0f99ab6e9f491d5eb3, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.941727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ef8bdc-01dd-48a4-b9b1-7e41a2fda704 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.957685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1363/2708] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:43 2022] PUT /identity/v3/projects/df73feb9212c4398a6528d6c183b0d29/users/6c328eb0742a4a0f99ab6e9f491d5eb3/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:43.968357 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724b41b-1cfe-4116-98a6-fbcaa963823c None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.968641 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724b41b-1cfe-4116-98a6-fbcaa963823c None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.968950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724b41b-1cfe-4116-98a6-fbcaa963823c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.999332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-1724b41b-1cfe-4116-98a6-fbcaa963823c None None] MFA Rules not processed for user `6c328eb0742a4a0f99ab6e9f491d5eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:44.000545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1724b41b-1cfe-4116-98a6-fbcaa963823c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:44.076559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1346/2709] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:44.087987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef8bdc-01dd-48a4-b9b1-7e41a2fda704 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:44.089846 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a0ef8bdc-01dd-48a4-b9b1-7e41a2fda704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:44.170874 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef8bdc-01dd-48a4-b9b1-7e41a2fda704 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:44.175084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30eed1-0a08-40cb-8316-a85a247f857b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:44.175690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30eed1-0a08-40cb-8316-a85a247f857b service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:44.175968 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30eed1-0a08-40cb-8316-a85a247f857b service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:44.176238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30eed1-0a08-40cb-8316-a85a247f857b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:44.185579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30eed1-0a08-40cb-8316-a85a247f857b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:44.187008 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30eed1-0a08-40cb-8316-a85a247f857b service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:44.196215 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1364/2710] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:47.646274 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724b41b-1cfe-4116-98a6-fbcaa963823c None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:47.647794 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-1724b41b-1cfe-4116-98a6-fbcaa963823c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:47.711439 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:47.730012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724b41b-1cfe-4116-98a6-fbcaa963823c None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:47.735386 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:47.736120 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:47.736463 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:47.736825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] PATH_INFO: `/v3/users/69cd8325f60f4eb7a1394b0caf06582a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:47.738005 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69cd8325f60f4eb7a1394b0caf06582a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:47.740232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:47.809684 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69cd8325f60f4eb7a1394b0caf06582a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:47.810338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69cd8325f60f4eb7a1394b0caf06582a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:47.812267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69cd8325f60f4eb7a1394b0caf06582a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:47.830434 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69cd8325f60f4eb7a1394b0caf06582a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:47.840545 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69cd8325f60f4eb7a1394b0caf06582a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:47.854147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1347/2711] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:47 2022] DELETE /identity/v3/users/69cd8325f60f4eb7a1394b0caf06582a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:47.859400 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c08671-98eb-44e7-949c-fa3a92c6c408 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:47.861140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c08671-98eb-44e7-949c-fa3a92c6c408 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:47.861611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c08671-98eb-44e7-949c-fa3a92c6c408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:47.887656 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-70c08671-98eb-44e7-949c-fa3a92c6c408 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:47.890065 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-70c08671-98eb-44e7-949c-fa3a92c6c408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:47.969636 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1365/2712] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:47.981575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:47.983785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:48.059406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14975ee-2da4-4c53-a134-809114fcb038 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:48.063812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f234ee-e8b1-4faf-b92e-f39c8e23f97c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:48.064362 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f234ee-e8b1-4faf-b92e-f39c8e23f97c service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:48.064746 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f234ee-e8b1-4faf-b92e-f39c8e23f97c service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:48.065029 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f234ee-e8b1-4faf-b92e-f39c8e23f97c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:48.075267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f234ee-e8b1-4faf-b92e-f39c8e23f97c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:48.077132 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f234ee-e8b1-4faf-b92e-f39c8e23f97c service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:48.089090 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1348/2713] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:48.359620 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c08671-98eb-44e7-949c-fa3a92c6c408 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:48.361365 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-70c08671-98eb-44e7-949c-fa3a92c6c408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:48.374713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c08671-98eb-44e7-949c-fa3a92c6c408 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:48.379701 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7c569-d2a6-4024-a579-ff0350f18d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:48.380378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7c569-d2a6-4024-a579-ff0350f18d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:48.380708 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7c569-d2a6-4024-a579-ff0350f18d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:48.380982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7c569-d2a6-4024-a579-ff0350f18d92 admin admin] PATH_INFO: `/v3/projects/bca754f41a1e4349a8268eba44d16d2c` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:48.382224 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7c569-d2a6-4024-a579-ff0350f18d92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bca754f41a1e4349a8268eba44d16d2c)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:48.384142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7c569-d2a6-4024-a579-ff0350f18d92 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:48.426602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:48.433913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7bd7c569-d2a6-4024-a579-ff0350f18d92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bca754f41a1e4349a8268eba44d16d2c', 'request_id': 'req-7bd7c569-d2a6-4024-a579-ff0350f18d92'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:48.443366 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7bd7c569-d2a6-4024-a579-ff0350f18d92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bca754f41a1e4349a8268eba44d16d2c', 'request_id': 'req-7bd7c569-d2a6-4024-a579-ff0350f18d92'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:48.450071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7bd7c569-d2a6-4024-a579-ff0350f18d92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bca754f41a1e4349a8268eba44d16d2c', 'request_id': 'req-7bd7c569-d2a6-4024-a579-ff0350f18d92'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:48.461595 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7bd7c569-d2a6-4024-a579-ff0350f18d92 admin admin] The token cache is being invalidate because project bca754f41a1e4349a8268eba44d16d2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:48.461883 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7bd7c569-d2a6-4024-a579-ff0350f18d92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:48.462516 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-7bd7c569-d2a6-4024-a579-ff0350f18d92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:48.464060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1366/2714] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:48 2022] DELETE /identity/v3/projects/bca754f41a1e4349a8268eba44d16d2c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:54.906166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f234ee-e8b1-4faf-b92e-f39c8e23f97c service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:54.907628 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-05f234ee-e8b1-4faf-b92e-f39c8e23f97c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:54.921528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f234ee-e8b1-4faf-b92e-f39c8e23f97c service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:54.989152 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:54.989831 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:54.990102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:54.990355 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] PATH_INFO: `/v3/users/6c328eb0742a4a0f99ab6e9f491d5eb3` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:54.991514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c328eb0742a4a0f99ab6e9f491d5eb3)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:54.993490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:55.006556 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:55.049257 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c328eb0742a4a0f99ab6e9f491d5eb3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:55.050062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c328eb0742a4a0f99ab6e9f491d5eb3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:55.050796 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c328eb0742a4a0f99ab6e9f491d5eb3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:55.065061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c328eb0742a4a0f99ab6e9f491d5eb3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:55.074546 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c328eb0742a4a0f99ab6e9f491d5eb3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:55.088210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1349/2715] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:54 2022] DELETE /identity/v3/users/6c328eb0742a4a0f99ab6e9f491d5eb3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:55.092478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ae7b5-470c-4ab8-a42f-330727ebebb2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:55.093087 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ae7b5-470c-4ab8-a42f-330727ebebb2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:55.093273 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ae7b5-470c-4ab8-a42f-330727ebebb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:55.118733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-fb9ae7b5-470c-4ab8-a42f-330727ebebb2 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:55.119983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fb9ae7b5-470c-4ab8-a42f-330727ebebb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:55.181560 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1367/2716] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:55.192097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:55.193702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:55.256612 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5502c8-0762-4044-96ca-d3e4161ee892 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:55.260570 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145b561-3d03-43e1-85b3-fc94b46d56a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:55.261221 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145b561-3d03-43e1-85b3-fc94b46d56a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:55.261522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145b561-3d03-43e1-85b3-fc94b46d56a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:55.261748 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145b561-3d03-43e1-85b3-fc94b46d56a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:55.271950 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145b561-3d03-43e1-85b3-fc94b46d56a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:55.273435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145b561-3d03-43e1-85b3-fc94b46d56a3 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:55.284614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1350/2717] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:55.673625 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ae7b5-470c-4ab8-a42f-330727ebebb2 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:55.675618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-fb9ae7b5-470c-4ab8-a42f-330727ebebb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:55.691902 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ae7b5-470c-4ab8-a42f-330727ebebb2 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:55.698043 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ebf7c-e064-4f93-8333-bb6244e611f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:55.698868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ebf7c-e064-4f93-8333-bb6244e611f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:55.699775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ebf7c-e064-4f93-8333-bb6244e611f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:55.699775 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ebf7c-e064-4f93-8333-bb6244e611f4 admin admin] PATH_INFO: `/v3/projects/df73feb9212c4398a6528d6c183b0d29` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:55.701465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ebf7c-e064-4f93-8333-bb6244e611f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df73feb9212c4398a6528d6c183b0d29)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:55.704976 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ebf7c-e064-4f93-8333-bb6244e611f4 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:55.770752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8d0ebf7c-e064-4f93-8333-bb6244e611f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df73feb9212c4398a6528d6c183b0d29', 'request_id': 'req-8d0ebf7c-e064-4f93-8333-bb6244e611f4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:55.785727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8d0ebf7c-e064-4f93-8333-bb6244e611f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df73feb9212c4398a6528d6c183b0d29', 'request_id': 'req-8d0ebf7c-e064-4f93-8333-bb6244e611f4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:55.793362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8d0ebf7c-e064-4f93-8333-bb6244e611f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df73feb9212c4398a6528d6c183b0d29', 'request_id': 'req-8d0ebf7c-e064-4f93-8333-bb6244e611f4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:55.805579 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8d0ebf7c-e064-4f93-8333-bb6244e611f4 admin admin] The token cache is being invalidate because project df73feb9212c4398a6528d6c183b0d29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:55.805965 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8d0ebf7c-e064-4f93-8333-bb6244e611f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:55.806698 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-8d0ebf7c-e064-4f93-8333-bb6244e611f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:55.808298 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1368/2718] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:55 2022] DELETE /identity/v3/projects/df73feb9212c4398a6528d6c183b0d29 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:55.813822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c150b7-86ce-4b25-897c-48ad64f41f55 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:55.814102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c150b7-86ce-4b25-897c-48ad64f41f55 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:55.814373 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c150b7-86ce-4b25-897c-48ad64f41f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:55.835880 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d2c150b7-86ce-4b25-897c-48ad64f41f55 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:55.837104 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d2c150b7-86ce-4b25-897c-48ad64f41f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:55.922178 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1351/2719] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:55.935486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ab7ce-1edc-4e28-9d18-182c9f55bb00 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:55.936135 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ab7ce-1edc-4e28-9d18-182c9f55bb00 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:55.936466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ab7ce-1edc-4e28-9d18-182c9f55bb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:55.957555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b85ab7ce-1edc-4e28-9d18-182c9f55bb00 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:55.958755 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b85ab7ce-1edc-4e28-9d18-182c9f55bb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:55.966501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1369/2720] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:55.971118 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c150b7-86ce-4b25-897c-48ad64f41f55 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:55.982414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c150b7-86ce-4b25-897c-48ad64f41f55 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:55.987667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f495620-8091-4821-b512-4c3c271d6fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:55.988391 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f495620-8091-4821-b512-4c3c271d6fbc admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:55.988718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f495620-8091-4821-b512-4c3c271d6fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:55.988997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f495620-8091-4821-b512-4c3c271d6fbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:55.989679 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f495620-8091-4821-b512-4c3c271d6fbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:55.989980 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f495620-8091-4821-b512-4c3c271d6fbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:55.991672 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f495620-8091-4821-b512-4c3c271d6fbc admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:56.004190 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1352/2721] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:56.008580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ab7ce-1edc-4e28-9d18-182c9f55bb00 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:56.018469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ab7ce-1edc-4e28-9d18-182c9f55bb00 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:56.023536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6d944-dd70-409b-bc58-4c90a134643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:56.024136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6d944-dd70-409b-bc58-4c90a134643f admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:56.024460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6d944-dd70-409b-bc58-4c90a134643f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:56.024702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6d944-dd70-409b-bc58-4c90a134643f admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:56.025381 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6d944-dd70-409b-bc58-4c90a134643f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:56.027318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6d944-dd70-409b-bc58-4c90a134643f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:56.058672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1370/2722] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:56 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:56.063277 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f495620-8091-4821-b512-4c3c271d6fbc admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:56.074153 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f495620-8091-4821-b512-4c3c271d6fbc admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:56.093157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:56.136355 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee650a-f123-46ee-9232-856b4fd7c2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:56.137027 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee650a-f123-46ee-9232-856b4fd7c2fb admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:56.137309 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee650a-f123-46ee-9232-856b4fd7c2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:56.137583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee650a-f123-46ee-9232-856b4fd7c2fb admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:56.138212 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee650a-f123-46ee-9232-856b4fd7c2fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:56.140183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee650a-f123-46ee-9232-856b4fd7c2fb admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:56.167540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1353/2723] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:56 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:56.174136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6d944-dd70-409b-bc58-4c90a134643f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:56.189388 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6d944-dd70-409b-bc58-4c90a134643f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:56.195095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7c3ba-e646-4656-9061-313e328c5b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:56.195692 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7c3ba-e646-4656-9061-313e328c5b62 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:56.195970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7c3ba-e646-4656-9061-313e328c5b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:56.196234 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7c3ba-e646-4656-9061-313e328c5b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:56.196916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7c3ba-e646-4656-9061-313e328c5b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:56.198534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7c3ba-e646-4656-9061-313e328c5b62 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:56.209678 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1371/2724] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:56.214528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee650a-f123-46ee-9232-856b4fd7c2fb admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:56.226744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee650a-f123-46ee-9232-856b4fd7c2fb admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:56.232665 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196949f-ad06-40a0-aa59-d999e99d9599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:56.233300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196949f-ad06-40a0-aa59-d999e99d9599 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:56.233575 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196949f-ad06-40a0-aa59-d999e99d9599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:56.233843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196949f-ad06-40a0-aa59-d999e99d9599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:56.234722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196949f-ad06-40a0-aa59-d999e99d9599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:56.236353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196949f-ad06-40a0-aa59-d999e99d9599 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:56.248551 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1354/2725] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:56.253233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7c3ba-e646-4656-9061-313e328c5b62 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:56.263943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7c3ba-e646-4656-9061-313e328c5b62 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:56.269997 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fb6a5-9f84-427e-8e1c-e1b6dddd4476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:56.270650 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fb6a5-9f84-427e-8e1c-e1b6dddd4476 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:56.270928 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fb6a5-9f84-427e-8e1c-e1b6dddd4476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:56.271200 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fb6a5-9f84-427e-8e1c-e1b6dddd4476 admin admin] PATH_INFO: `/v3/projects/121d2511f42847cab3930da359a3a7ff/users/85620ae3c59a44da898ec76436271027/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:56.283745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:56.289938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fb6a5-9f84-427e-8e1c-e1b6dddd4476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=121d2511f42847cab3930da359a3a7ff, user_id=85620ae3c59a44da898ec76436271027, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:56.294501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fb6a5-9f84-427e-8e1c-e1b6dddd4476 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:56.312405 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1372/2726] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:56 2022] PUT /identity/v3/projects/121d2511f42847cab3930da359a3a7ff/users/85620ae3c59a44da898ec76436271027/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:56.317570 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6831727-1faf-4422-ae8f-f6916b1f1fd3 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:56.317847 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6831727-1faf-4422-ae8f-f6916b1f1fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:56.318114 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6831727-1faf-4422-ae8f-f6916b1f1fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:56.338145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c6831727-1faf-4422-ae8f-f6916b1f1fd3 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:56.339930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c6831727-1faf-4422-ae8f-f6916b1f1fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:56.412985 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1355/2727] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:56.423752 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fb6a5-9f84-427e-8e1c-e1b6dddd4476 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:56.425940 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-687fb6a5-9f84-427e-8e1c-e1b6dddd4476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:56.487766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fb6a5-9f84-427e-8e1c-e1b6dddd4476 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:56.491810 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea80cbc-80a9-4e74-8d69-47a205feedb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:56.492449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea80cbc-80a9-4e74-8d69-47a205feedb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:56.492754 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea80cbc-80a9-4e74-8d69-47a205feedb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:56.493031 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea80cbc-80a9-4e74-8d69-47a205feedb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:56.502648 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea80cbc-80a9-4e74-8d69-47a205feedb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:56.504098 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea80cbc-80a9-4e74-8d69-47a205feedb5 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:56.513750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1373/2728] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:57.216228 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1d10e-1b00-4516-9a93-83f77f0d7b66 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:57.216807 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1d10e-1b00-4516-9a93-83f77f0d7b66 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:57.216807 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1d10e-1b00-4516-9a93-83f77f0d7b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:57.249298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c7b1d10e-1b00-4516-9a93-83f77f0d7b66 None None] MFA Rules not processed for user `85620ae3c59a44da898ec76436271027`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:57.250540 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c7b1d10e-1b00-4516-9a93-83f77f0d7b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:57.373935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1356/2729] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:57.374792 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea80cbc-80a9-4e74-8d69-47a205feedb5 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:57.377131 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7ea80cbc-80a9-4e74-8d69-47a205feedb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:57.449124 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea80cbc-80a9-4e74-8d69-47a205feedb5 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:57.454360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcb180-4e55-4495-998f-e2a7de9fca32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:57.455223 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcb180-4e55-4495-998f-e2a7de9fca32 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:57.455608 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcb180-4e55-4495-998f-e2a7de9fca32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:57.455970 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcb180-4e55-4495-998f-e2a7de9fca32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:57.470710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcb180-4e55-4495-998f-e2a7de9fca32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:57.473120 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcb180-4e55-4495-998f-e2a7de9fca32 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:57.486105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1374/2730] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:40:58.870158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd8fd8-3184-4c4d-85c5-1ce300a92e89 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:58.870691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd8fd8-3184-4c4d-85c5-1ce300a92e89 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:58.870691 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd8fd8-3184-4c4d-85c5-1ce300a92e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:58.890256 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-ddcd8fd8-3184-4c4d-85c5-1ce300a92e89 None None] MFA Rules not processed for user `85620ae3c59a44da898ec76436271027`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:58.892197 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ddcd8fd8-3184-4c4d-85c5-1ce300a92e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:58.899921 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1357/2731] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:40:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:07.480356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=56,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:08.885239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:23.088252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcb180-4e55-4495-998f-e2a7de9fca32 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:23.089923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e8dcb180-4e55-4495-998f-e2a7de9fca32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:23.105332 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcb180-4e55-4495-998f-e2a7de9fca32 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:23.110328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:23.111029 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:23.111309 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:23.111582 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] PATH_INFO: `/v3/users/86f7e2ec66c048b8a3952a9328ccd237` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:23.112866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f7e2ec66c048b8a3952a9328ccd237)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:23.118029 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:23.172155 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f7e2ec66c048b8a3952a9328ccd237'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.172842 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f7e2ec66c048b8a3952a9328ccd237'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.173444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f7e2ec66c048b8a3952a9328ccd237'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.186318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f7e2ec66c048b8a3952a9328ccd237'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.195429 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f7e2ec66c048b8a3952a9328ccd237'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.210134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1375/2732] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:41:23 2022] DELETE /identity/v3/users/86f7e2ec66c048b8a3952a9328ccd237 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:23.214715 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315167d-71a2-4218-83b5-a84255c3d7d4 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:23.215266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315167d-71a2-4218-83b5-a84255c3d7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:23.215580 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315167d-71a2-4218-83b5-a84255c3d7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:23.236117 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-8315167d-71a2-4218-83b5-a84255c3d7d4 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:23.237370 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8315167d-71a2-4218-83b5-a84255c3d7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:23.301792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1358/2733] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:23.311424 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:23.312979 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:23.400660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44159d3-d0aa-481a-a539-6b78ae8f9976 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:23.405443 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f2383-e670-4f1d-9b22-e4902fe7b532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:23.406383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f2383-e670-4f1d-9b22-e4902fe7b532 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:23.406688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f2383-e670-4f1d-9b22-e4902fe7b532 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:23.406934 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f2383-e670-4f1d-9b22-e4902fe7b532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:23.417536 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f2383-e670-4f1d-9b22-e4902fe7b532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:23.419362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f2383-e670-4f1d-9b22-e4902fe7b532 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:23.430258 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1376/2734] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:41:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:41:23.709326 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315167d-71a2-4218-83b5-a84255c3d7d4 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:23.710870 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-8315167d-71a2-4218-83b5-a84255c3d7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:23.723178 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315167d-71a2-4218-83b5-a84255c3d7d4 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:23.730218 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:23.730896 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:23.731154 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:23.731430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] PATH_INFO: `/v3/projects/7a0cc4dc0928432aaba22a58e9401393` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:23.732970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a0cc4dc0928432aaba22a58e9401393)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:23.734917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:23.788767 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a0cc4dc0928432aaba22a58e9401393', 'request_id': 'req-82a84273-d8ca-49e4-bf58-0de469ff0be3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.801871 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a0cc4dc0928432aaba22a58e9401393', 'request_id': 'req-82a84273-d8ca-49e4-bf58-0de469ff0be3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.809763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a0cc4dc0928432aaba22a58e9401393', 'request_id': 'req-82a84273-d8ca-49e4-bf58-0de469ff0be3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.822804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] The token cache is being invalidate because project 7a0cc4dc0928432aaba22a58e9401393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:41:23.823246 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.824031 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.825711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1359/2735] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:41:23 2022] DELETE /identity/v3/projects/7a0cc4dc0928432aaba22a58e9401393 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:23.829819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f2383-e670-4f1d-9b22-e4902fe7b532 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:23.841828 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f2383-e670-4f1d-9b22-e4902fe7b532 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:23.904710 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:23.913164 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:23.913882 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:23.914235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:23.914754 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] PATH_INFO: `/v3/users/66aa92edeec448aebeef72ae6987e8ad` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:23.916243 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=66aa92edeec448aebeef72ae6987e8ad)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:23.918489 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:23.983271 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66aa92edeec448aebeef72ae6987e8ad'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.984207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66aa92edeec448aebeef72ae6987e8ad'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.985552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66aa92edeec448aebeef72ae6987e8ad'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:24.001237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66aa92edeec448aebeef72ae6987e8ad'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:24.005206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:24.006795 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:24.011614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66aa92edeec448aebeef72ae6987e8ad'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:24.033545 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1377/2736] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:41:23 2022] DELETE /identity/v3/users/66aa92edeec448aebeef72ae6987e8ad => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:41:24.117424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a84273-d8ca-49e4-bf58-0de469ff0be3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:24.121055 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:24.183906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf463d-6858-4718-bef2-149210698406 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:24.184528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf463d-6858-4718-bef2-149210698406 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:24.184874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf463d-6858-4718-bef2-149210698406 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:24.185173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf463d-6858-4718-bef2-149210698406 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:24.187009 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-44cf463d-6858-4718-bef2-149210698406 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:24.204150 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf463d-6858-4718-bef2-149210698406 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:24.205744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf463d-6858-4718-bef2-149210698406 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:24.218957 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1360/2737] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:41:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:41:24.319750 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:24.321655 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:24.399776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41095723-1f32-4a1d-9891-0f6063511f1e admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:24.408206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec247d-3df3-425f-85e4-387722ee3614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:24.409442 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec247d-3df3-425f-85e4-387722ee3614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:24.409937 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec247d-3df3-425f-85e4-387722ee3614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:24.410671 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec247d-3df3-425f-85e4-387722ee3614 admin admin] PATH_INFO: `/v3/projects/0d698e62c91e4837a1e624b93b16937d` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:24.415297 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec247d-3df3-425f-85e4-387722ee3614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d698e62c91e4837a1e624b93b16937d)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:24.417517 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec247d-3df3-425f-85e4-387722ee3614 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:24.472002 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-81ec247d-3df3-425f-85e4-387722ee3614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d698e62c91e4837a1e624b93b16937d', 'request_id': 'req-81ec247d-3df3-425f-85e4-387722ee3614'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:24.481863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-81ec247d-3df3-425f-85e4-387722ee3614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d698e62c91e4837a1e624b93b16937d', 'request_id': 'req-81ec247d-3df3-425f-85e4-387722ee3614'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:24.489895 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-81ec247d-3df3-425f-85e4-387722ee3614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d698e62c91e4837a1e624b93b16937d', 'request_id': 'req-81ec247d-3df3-425f-85e4-387722ee3614'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:24.501770 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-81ec247d-3df3-425f-85e4-387722ee3614 admin admin] The token cache is being invalidate because project 0d698e62c91e4837a1e624b93b16937d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:41:24.502062 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-81ec247d-3df3-425f-85e4-387722ee3614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:24.502748 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-81ec247d-3df3-425f-85e4-387722ee3614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:24.505158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1378/2738] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:41:24 2022] DELETE /identity/v3/projects/0d698e62c91e4837a1e624b93b16937d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:24.510195 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc25a0-24e3-473f-9148-ce33c9d5a039 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:24.510473 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc25a0-24e3-473f-9148-ce33c9d5a039 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:24.510767 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc25a0-24e3-473f-9148-ce33c9d5a039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:24.532338 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-63cc25a0-24e3-473f-9148-ce33c9d5a039 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:24.533585 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-63cc25a0-24e3-473f-9148-ce33c9d5a039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:24.597471 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1361/2739] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:24.605713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5633d-889b-4eae-a03c-f49cea2af248 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:24.606003 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5633d-889b-4eae-a03c-f49cea2af248 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:24.606285 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5633d-889b-4eae-a03c-f49cea2af248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:24.615588 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:24.634846 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-0ec5633d-889b-4eae-a03c-f49cea2af248 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:24.636075 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-0ec5633d-889b-4eae-a03c-f49cea2af248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:24.646262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1379/2740] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:24.649101 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc25a0-24e3-473f-9148-ce33c9d5a039 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:24.660981 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc25a0-24e3-473f-9148-ce33c9d5a039 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:24.668383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263856ce-bb62-48fd-8ae7-bd7209f5b7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:24.669954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263856ce-bb62-48fd-8ae7-bd7209f5b7f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:24.670191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263856ce-bb62-48fd-8ae7-bd7209f5b7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:24.670782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263856ce-bb62-48fd-8ae7-bd7209f5b7f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:24.671447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263856ce-bb62-48fd-8ae7-bd7209f5b7f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:41:24.671797 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263856ce-bb62-48fd-8ae7-bd7209f5b7f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:24.673439 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263856ce-bb62-48fd-8ae7-bd7209f5b7f9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:24.689144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1362/2741] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:41:24.693585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5633d-889b-4eae-a03c-f49cea2af248 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:24.703796 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5633d-889b-4eae-a03c-f49cea2af248 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:24.709190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bd1d4-d02a-440d-b3c8-70af3d9fdddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:24.709866 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bd1d4-d02a-440d-b3c8-70af3d9fdddd admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:24.710140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bd1d4-d02a-440d-b3c8-70af3d9fdddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:24.711080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bd1d4-d02a-440d-b3c8-70af3d9fdddd admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:24.712306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bd1d4-d02a-440d-b3c8-70af3d9fdddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:24.714543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bd1d4-d02a-440d-b3c8-70af3d9fdddd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:24.753123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1380/2742] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:41:24 2022] POST /identity/v3/projects => generated 363 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:41:24.757296 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263856ce-bb62-48fd-8ae7-bd7209f5b7f9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:24.769952 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263856ce-bb62-48fd-8ae7-bd7209f5b7f9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:24.831855 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241dd0e-626f-494c-b609-ebd90b2843e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:24.832778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241dd0e-626f-494c-b609-ebd90b2843e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:24.833148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241dd0e-626f-494c-b609-ebd90b2843e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:24.833282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241dd0e-626f-494c-b609-ebd90b2843e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:24.833904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241dd0e-626f-494c-b609-ebd90b2843e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:24.835851 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241dd0e-626f-494c-b609-ebd90b2843e6 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:24.866068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1363/2743] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:41:24 2022] POST /identity/v3/users => generated 282 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:41:24.867677 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bd1d4-d02a-440d-b3c8-70af3d9fdddd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:24.878469 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bd1d4-d02a-440d-b3c8-70af3d9fdddd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:24.883324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0c2bf-a2fa-49e4-a869-7fdc3925b6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:24.883966 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0c2bf-a2fa-49e4-a869-7fdc3925b6bd admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:24.884235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0c2bf-a2fa-49e4-a869-7fdc3925b6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:24.884496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0c2bf-a2fa-49e4-a869-7fdc3925b6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:24.885300 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0c2bf-a2fa-49e4-a869-7fdc3925b6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:24.886926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0c2bf-a2fa-49e4-a869-7fdc3925b6bd admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:24.898083 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1381/2744] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:41:24.902201 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241dd0e-626f-494c-b609-ebd90b2843e6 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:24.913587 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241dd0e-626f-494c-b609-ebd90b2843e6 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:24.919975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529f177-56b3-4171-881d-1e30248f60c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:24.920589 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529f177-56b3-4171-881d-1e30248f60c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:24.920898 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529f177-56b3-4171-881d-1e30248f60c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:24.921168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529f177-56b3-4171-881d-1e30248f60c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:24.921821 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529f177-56b3-4171-881d-1e30248f60c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:24.923447 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529f177-56b3-4171-881d-1e30248f60c1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:24.935309 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1364/2745] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:41:24.940287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0c2bf-a2fa-49e4-a869-7fdc3925b6bd admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:24.956766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0c2bf-a2fa-49e4-a869-7fdc3925b6bd admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:24.961747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739f9fb-c1cd-483f-bab2-5e000d9ffde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:24.962395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739f9fb-c1cd-483f-bab2-5e000d9ffde2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:24.962672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739f9fb-c1cd-483f-bab2-5e000d9ffde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:24.963067 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739f9fb-c1cd-483f-bab2-5e000d9ffde2 admin admin] PATH_INFO: `/v3/projects/681ba3dfeedd4bde809873d3bd8f46de/users/173a9a5133a54b39849de93dd798fa28/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:24.980847 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739f9fb-c1cd-483f-bab2-5e000d9ffde2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=681ba3dfeedd4bde809873d3bd8f46de, user_id=173a9a5133a54b39849de93dd798fa28, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:24.983440 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739f9fb-c1cd-483f-bab2-5e000d9ffde2 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:24.999119 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1382/2746] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:41:24 2022] PUT /identity/v3/projects/681ba3dfeedd4bde809873d3bd8f46de/users/173a9a5133a54b39849de93dd798fa28/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:41:25.010555 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566edca-27bc-4080-b2e3-4857e3426e88 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:25.010769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566edca-27bc-4080-b2e3-4857e3426e88 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:25.011044 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566edca-27bc-4080-b2e3-4857e3426e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:25.019448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:25.047062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-b566edca-27bc-4080-b2e3-4857e3426e88 None None] MFA Rules not processed for user `173a9a5133a54b39849de93dd798fa28`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:25.048726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-b566edca-27bc-4080-b2e3-4857e3426e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:25.131302 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5818c-a411-4302-9dc0-94339117c4cd None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:25.131690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5818c-a411-4302-9dc0-94339117c4cd None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:25.131886 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5818c-a411-4302-9dc0-94339117c4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:25.143088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1365/2747] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:41:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:25.147726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566edca-27bc-4080-b2e3-4857e3426e88 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:25.164838 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-27c5818c-a411-4302-9dc0-94339117c4cd None None] MFA Rules not processed for user `ae1cd97c879141e2b4c1527b5964d054`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:25.166964 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-27c5818c-a411-4302-9dc0-94339117c4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:25.237026 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566edca-27bc-4080-b2e3-4857e3426e88 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:25.244759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f9587-03dd-4546-ac94-51f4432e6a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:25.244759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f9587-03dd-4546-ac94-51f4432e6a8b admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:25.244759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f9587-03dd-4546-ac94-51f4432e6a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:25.244759 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f9587-03dd-4546-ac94-51f4432e6a8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:25.245421 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f9587-03dd-4546-ac94-51f4432e6a8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:25.246942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f9587-03dd-4546-ac94-51f4432e6a8b admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:25.249454 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1383/2748] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:41:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:41:25.292860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1366/2749] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:41:25 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:41:25.294103 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5818c-a411-4302-9dc0-94339117c4cd None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:25.312686 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5818c-a411-4302-9dc0-94339117c4cd None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:25.398030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d78dc8-97af-4543-bbaa-a42d5f7ef4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:25.398956 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78dc8-97af-4543-bbaa-a42d5f7ef4bf admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:25.399235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78dc8-97af-4543-bbaa-a42d5f7ef4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:25.399570 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78dc8-97af-4543-bbaa-a42d5f7ef4bf admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:25.404158 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d78dc8-97af-4543-bbaa-a42d5f7ef4bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:25.406486 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d78dc8-97af-4543-bbaa-a42d5f7ef4bf admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:25.433489 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1384/2750] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:41:25 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:41:25.434977 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f9587-03dd-4546-ac94-51f4432e6a8b admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:25.446925 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f9587-03dd-4546-ac94-51f4432e6a8b admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:25.451863 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af3cba-86d5-4648-9c1e-c1ca2f5601a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:25.452560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af3cba-86d5-4648-9c1e-c1ca2f5601a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:25.452874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af3cba-86d5-4648-9c1e-c1ca2f5601a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:25.453147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af3cba-86d5-4648-9c1e-c1ca2f5601a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:25.453804 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af3cba-86d5-4648-9c1e-c1ca2f5601a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:25.455441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af3cba-86d5-4648-9c1e-c1ca2f5601a1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:25.466353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1367/2751] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:41:25.470738 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d78dc8-97af-4543-bbaa-a42d5f7ef4bf admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:25.481611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d78dc8-97af-4543-bbaa-a42d5f7ef4bf admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:25.494336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff8350-5de8-4228-a8a8-43991702df31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:25.495094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff8350-5de8-4228-a8a8-43991702df31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:25.495398 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff8350-5de8-4228-a8a8-43991702df31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:25.495668 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff8350-5de8-4228-a8a8-43991702df31 admin admin] PATH_INFO: `/v3/projects/5f8f1a86e62f43b59103586732bf0d08/users/435ac9fc6cb640078e3d3ab6afb5ab73/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:25.515876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff8350-5de8-4228-a8a8-43991702df31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8f1a86e62f43b59103586732bf0d08, user_id=435ac9fc6cb640078e3d3ab6afb5ab73, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:25.518618 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff8350-5de8-4228-a8a8-43991702df31 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:25.539503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1385/2752] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:41:25 2022] PUT /identity/v3/projects/5f8f1a86e62f43b59103586732bf0d08/users/435ac9fc6cb640078e3d3ab6afb5ab73/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:41:25.543760 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af3cba-86d5-4648-9c1e-c1ca2f5601a1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:25.557884 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af3cba-86d5-4648-9c1e-c1ca2f5601a1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:25.630424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866922e-3b6c-4630-8f82-4c3b44b19a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:25.631062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866922e-3b6c-4630-8f82-4c3b44b19a8c admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:25.631312 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866922e-3b6c-4630-8f82-4c3b44b19a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:25.631601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866922e-3b6c-4630-8f82-4c3b44b19a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:25.632257 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c866922e-3b6c-4630-8f82-4c3b44b19a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:25.634726 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c866922e-3b6c-4630-8f82-4c3b44b19a8c admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:25.650393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1368/2753] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:41:25.655746 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff8350-5de8-4228-a8a8-43991702df31 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:25.670727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff8350-5de8-4228-a8a8-43991702df31 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:25.676686 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d4716-e92e-4a81-8fa4-2e5dee173807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:25.677774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d4716-e92e-4a81-8fa4-2e5dee173807 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:25.678240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d4716-e92e-4a81-8fa4-2e5dee173807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:25.678701 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d4716-e92e-4a81-8fa4-2e5dee173807 admin admin] PATH_INFO: `/v3/projects/5f8f1a86e62f43b59103586732bf0d08/users/435ac9fc6cb640078e3d3ab6afb5ab73/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:25.680879 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d4716-e92e-4a81-8fa4-2e5dee173807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8f1a86e62f43b59103586732bf0d08, user_id=435ac9fc6cb640078e3d3ab6afb5ab73, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:25.685922 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d4716-e92e-4a81-8fa4-2e5dee173807 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:25.693447 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:25.700056 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1386/2754] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:41:25 2022] PUT /identity/v3/projects/5f8f1a86e62f43b59103586732bf0d08/users/435ac9fc6cb640078e3d3ab6afb5ab73/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:25.713417 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789b285-03e9-401f-a6ab-06cd4869cefa None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:25.714411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789b285-03e9-401f-a6ab-06cd4869cefa None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:25.715068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789b285-03e9-401f-a6ab-06cd4869cefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:25.726906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:25.750606 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-2789b285-03e9-401f-a6ab-06cd4869cefa None None] MFA Rules not processed for user `435ac9fc6cb640078e3d3ab6afb5ab73`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:25.752196 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2789b285-03e9-401f-a6ab-06cd4869cefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:25.838482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1369/2755] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:41:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:25.852492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d4716-e92e-4a81-8fa4-2e5dee173807 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:25.854856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-547d4716-e92e-4a81-8fa4-2e5dee173807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:25.964915 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d4716-e92e-4a81-8fa4-2e5dee173807 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:25.970697 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270e6d7-b48d-41c0-ad91-c438ec329ba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:25.972456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270e6d7-b48d-41c0-ad91-c438ec329ba0 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:25.972876 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270e6d7-b48d-41c0-ad91-c438ec329ba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:25.973216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270e6d7-b48d-41c0-ad91-c438ec329ba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:25.985929 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270e6d7-b48d-41c0-ad91-c438ec329ba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:25.987799 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270e6d7-b48d-41c0-ad91-c438ec329ba0 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:25.998901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1387/2756] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:41:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:41:26.090072 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789b285-03e9-401f-a6ab-06cd4869cefa None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:26.100183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789b285-03e9-401f-a6ab-06cd4869cefa None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:26.104526 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1a09d-caac-4363-86c6-dfedc75df9fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:26.105172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1a09d-caac-4363-86c6-dfedc75df9fb service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:26.105457 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1a09d-caac-4363-86c6-dfedc75df9fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:26.106407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1a09d-caac-4363-86c6-dfedc75df9fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:26.160709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185c44-cb43-4783-9c34-681a5565ea7e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:26.161357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185c44-cb43-4783-9c34-681a5565ea7e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:26.161357 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185c44-cb43-4783-9c34-681a5565ea7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:26.181459 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-1f185c44-cb43-4783-9c34-681a5565ea7e None None] MFA Rules not processed for user `85620ae3c59a44da898ec76436271027`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:26.182610 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1f185c44-cb43-4783-9c34-681a5565ea7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:26.200189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1a09d-caac-4363-86c6-dfedc75df9fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:26.201793 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1a09d-caac-4363-86c6-dfedc75df9fb service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:26.211801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1370/2757] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:41:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:41:26.266687 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1388/2758] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:41:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:35.636666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1a09d-caac-4363-86c6-dfedc75df9fb service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:35.714935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1a09d-caac-4363-86c6-dfedc75df9fb service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:35.720415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:35.721074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:35.721346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:35.721616 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] PATH_INFO: `/v3/users/173a9a5133a54b39849de93dd798fa28` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:35.722858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=173a9a5133a54b39849de93dd798fa28)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:35.724911 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:35.887661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:35.909300 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '173a9a5133a54b39849de93dd798fa28'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:35.909995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '173a9a5133a54b39849de93dd798fa28'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:35.910612 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '173a9a5133a54b39849de93dd798fa28'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:35.925533 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '173a9a5133a54b39849de93dd798fa28'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:35.936430 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '173a9a5133a54b39849de93dd798fa28'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:35.950194 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1371/2759] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:41:35 2022] DELETE /identity/v3/users/173a9a5133a54b39849de93dd798fa28 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:35.954768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ad1e-01a8-46e6-a409-86745263a5af None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:35.955266 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ad1e-01a8-46e6-a409-86745263a5af None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:35.955380 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ad1e-01a8-46e6-a409-86745263a5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:35.976229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7ca6ad1e-01a8-46e6-a409-86745263a5af None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:35.977508 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7ca6ad1e-01a8-46e6-a409-86745263a5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:36.043943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1389/2760] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:36.054465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:36.056050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:36.122327 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529103bd-7b5c-41f5-a969-5e71c8a2fb2e admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:36.128384 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956e654-7380-4c70-9f83-65cb1d4ce79b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:36.129458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956e654-7380-4c70-9f83-65cb1d4ce79b service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:36.129961 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956e654-7380-4c70-9f83-65cb1d4ce79b service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:36.130500 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956e654-7380-4c70-9f83-65cb1d4ce79b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:36.147008 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956e654-7380-4c70-9f83-65cb1d4ce79b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:36.148483 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956e654-7380-4c70-9f83-65cb1d4ce79b service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:36.159088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1372/2761] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:41:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:41:36.453951 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6ad1e-01a8-46e6-a409-86745263a5af None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:36.456422 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7ca6ad1e-01a8-46e6-a409-86745263a5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:36.475195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6ad1e-01a8-46e6-a409-86745263a5af None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:36.483033 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:36.484025 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:36.484283 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:36.484556 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] PATH_INFO: `/v3/projects/681ba3dfeedd4bde809873d3bd8f46de` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:36.485941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] RBAC: Authorizing `identity:delete_project(project_id=681ba3dfeedd4bde809873d3bd8f46de)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:36.488328 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:36.521747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:36.556321 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681ba3dfeedd4bde809873d3bd8f46de', 'request_id': 'req-1d97f79f-ff9f-4936-9744-fa6291dda55c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:36.565903 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681ba3dfeedd4bde809873d3bd8f46de', 'request_id': 'req-1d97f79f-ff9f-4936-9744-fa6291dda55c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:36.574162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681ba3dfeedd4bde809873d3bd8f46de', 'request_id': 'req-1d97f79f-ff9f-4936-9744-fa6291dda55c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:36.586709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] The token cache is being invalidate because project 681ba3dfeedd4bde809873d3bd8f46de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:41:36.587020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:36.587715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:36.589467 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1390/2762] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:41:36 2022] DELETE /identity/v3/projects/681ba3dfeedd4bde809873d3bd8f46de => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:36.593307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956e654-7380-4c70-9f83-65cb1d4ce79b service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:36.607440 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956e654-7380-4c70-9f83-65cb1d4ce79b service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:36.670803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53f1c2-a6e8-4c4b-a912-8e1cc7349310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:36.671519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53f1c2-a6e8-4c4b-a912-8e1cc7349310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:36.671698 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53f1c2-a6e8-4c4b-a912-8e1cc7349310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:36.671967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53f1c2-a6e8-4c4b-a912-8e1cc7349310 admin admin] PATH_INFO: `/v3/users/435ac9fc6cb640078e3d3ab6afb5ab73` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:36.673232 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53f1c2-a6e8-4c4b-a912-8e1cc7349310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=435ac9fc6cb640078e3d3ab6afb5ab73)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:36.675199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53f1c2-a6e8-4c4b-a912-8e1cc7349310 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:36.730209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fb53f1c2-a6e8-4c4b-a912-8e1cc7349310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '435ac9fc6cb640078e3d3ab6afb5ab73'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:36.730875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fb53f1c2-a6e8-4c4b-a912-8e1cc7349310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '435ac9fc6cb640078e3d3ab6afb5ab73'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:36.731529 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fb53f1c2-a6e8-4c4b-a912-8e1cc7349310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '435ac9fc6cb640078e3d3ab6afb5ab73'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:36.734266 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:36.750107 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fb53f1c2-a6e8-4c4b-a912-8e1cc7349310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '435ac9fc6cb640078e3d3ab6afb5ab73'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:36.758995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-fb53f1c2-a6e8-4c4b-a912-8e1cc7349310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '435ac9fc6cb640078e3d3ab6afb5ab73'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:36.771632 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1373/2763] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:41:36 2022] DELETE /identity/v3/users/435ac9fc6cb640078e3d3ab6afb5ab73 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:37.053271 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:37.054779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:37.068282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97f79f-ff9f-4936-9744-fa6291dda55c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:37.127848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3175c0-658f-480a-919a-796225676c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:37.128492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3175c0-658f-480a-919a-796225676c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:37.128875 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3175c0-658f-480a-919a-796225676c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:37.129152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3175c0-658f-480a-919a-796225676c69 admin admin] PATH_INFO: `/v3/projects/5f8f1a86e62f43b59103586732bf0d08` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:37.130401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3175c0-658f-480a-919a-796225676c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f8f1a86e62f43b59103586732bf0d08)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:37.132264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3175c0-658f-480a-919a-796225676c69 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:37.189768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bc3175c0-658f-480a-919a-796225676c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f8f1a86e62f43b59103586732bf0d08', 'request_id': 'req-bc3175c0-658f-480a-919a-796225676c69'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:37.199299 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bc3175c0-658f-480a-919a-796225676c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f8f1a86e62f43b59103586732bf0d08', 'request_id': 'req-bc3175c0-658f-480a-919a-796225676c69'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:37.206235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bc3175c0-658f-480a-919a-796225676c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f8f1a86e62f43b59103586732bf0d08', 'request_id': 'req-bc3175c0-658f-480a-919a-796225676c69'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:37.217634 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bc3175c0-658f-480a-919a-796225676c69 admin admin] The token cache is being invalidate because project 5f8f1a86e62f43b59103586732bf0d08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:41:37.217921 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bc3175c0-658f-480a-919a-796225676c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:37.218589 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bc3175c0-658f-480a-919a-796225676c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:37.220089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1391/2764] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:41:37 2022] DELETE /identity/v3/projects/5f8f1a86e62f43b59103586732bf0d08 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:37.225761 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedefcb9-2bbe-4622-a82d-a01add50e513 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:37.226095 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedefcb9-2bbe-4622-a82d-a01add50e513 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:37.226414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedefcb9-2bbe-4622-a82d-a01add50e513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:37.248137 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-cedefcb9-2bbe-4622-a82d-a01add50e513 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:37.249383 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-cedefcb9-2bbe-4622-a82d-a01add50e513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:37.314195 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1374/2765] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:37.322720 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49608f9-8328-40e1-90a8-3732fe17ec68 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:37.323010 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49608f9-8328-40e1-90a8-3732fe17ec68 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:37.323282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49608f9-8328-40e1-90a8-3732fe17ec68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:37.345745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b49608f9-8328-40e1-90a8-3732fe17ec68 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:37.346980 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b49608f9-8328-40e1-90a8-3732fe17ec68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:37.356623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1392/2766] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:37.361773 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedefcb9-2bbe-4622-a82d-a01add50e513 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:37.377018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedefcb9-2bbe-4622-a82d-a01add50e513 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:37.383998 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0d0e5-68e5-4de5-be2a-72bee1be5ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:37.384610 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0d0e5-68e5-4de5-be2a-72bee1be5ebe admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:37.384898 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0d0e5-68e5-4de5-be2a-72bee1be5ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:37.385166 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0d0e5-68e5-4de5-be2a-72bee1be5ebe admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:37.385812 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0d0e5-68e5-4de5-be2a-72bee1be5ebe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:41:37.386110 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0d0e5-68e5-4de5-be2a-72bee1be5ebe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:37.387754 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0d0e5-68e5-4de5-be2a-72bee1be5ebe admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:37.400535 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1375/2767] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:41:37.405216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49608f9-8328-40e1-90a8-3732fe17ec68 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:37.415783 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49608f9-8328-40e1-90a8-3732fe17ec68 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:37.422092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56948-5eb6-4852-ae10-ee8d9f92af98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:37.422694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56948-5eb6-4852-ae10-ee8d9f92af98 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:37.422972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56948-5eb6-4852-ae10-ee8d9f92af98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:37.423241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56948-5eb6-4852-ae10-ee8d9f92af98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:37.423906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f56948-5eb6-4852-ae10-ee8d9f92af98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:37.425884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f56948-5eb6-4852-ae10-ee8d9f92af98 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:37.457315 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1393/2768] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:41:37 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:41:37.462292 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0d0e5-68e5-4de5-be2a-72bee1be5ebe admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:37.472542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0d0e5-68e5-4de5-be2a-72bee1be5ebe admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:37.542063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7a731-d331-4c17-9295-2aa31d251dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:37.542671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7a731-d331-4c17-9295-2aa31d251dac admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:37.542945 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7a731-d331-4c17-9295-2aa31d251dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:37.543216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7a731-d331-4c17-9295-2aa31d251dac admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:37.543849 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7a731-d331-4c17-9295-2aa31d251dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:37.545877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7a731-d331-4c17-9295-2aa31d251dac admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:37.570167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1376/2769] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:41:37 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:41:37.575910 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56948-5eb6-4852-ae10-ee8d9f92af98 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:37.587022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56948-5eb6-4852-ae10-ee8d9f92af98 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:37.592154 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19f3ba-26fc-462c-9045-bae888d9527d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:37.592788 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19f3ba-26fc-462c-9045-bae888d9527d admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:37.593063 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19f3ba-26fc-462c-9045-bae888d9527d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:37.593334 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19f3ba-26fc-462c-9045-bae888d9527d admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:37.593962 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19f3ba-26fc-462c-9045-bae888d9527d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:37.595584 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19f3ba-26fc-462c-9045-bae888d9527d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:37.601117 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:37.606483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1394/2770] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:41:37.611620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7a731-d331-4c17-9295-2aa31d251dac admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:37.622646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7a731-d331-4c17-9295-2aa31d251dac admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:37.628454 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb283d6a-f36f-4255-8355-89e0fdfc6f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:37.629183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb283d6a-f36f-4255-8355-89e0fdfc6f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:37.629465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb283d6a-f36f-4255-8355-89e0fdfc6f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:37.629736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb283d6a-f36f-4255-8355-89e0fdfc6f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:37.630427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb283d6a-f36f-4255-8355-89e0fdfc6f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:37.632158 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb283d6a-f36f-4255-8355-89e0fdfc6f18 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:37.642778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1377/2771] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:41:37.647314 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19f3ba-26fc-462c-9045-bae888d9527d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:37.657781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19f3ba-26fc-462c-9045-bae888d9527d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:37.663665 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81bc07-a6f5-49aa-bd2b-071b3bd7130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:37.664352 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81bc07-a6f5-49aa-bd2b-071b3bd7130c admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:37.664625 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81bc07-a6f5-49aa-bd2b-071b3bd7130c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:37.664923 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81bc07-a6f5-49aa-bd2b-071b3bd7130c admin admin] PATH_INFO: `/v3/projects/95a49c5324904115986e3755f5f4d781/users/f00e97b9015d48bb96afd474bf86c1b8/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:37.681465 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81bc07-a6f5-49aa-bd2b-071b3bd7130c admin admin] RBAC: Authorizing `identity:create_grant(project_id=95a49c5324904115986e3755f5f4d781, user_id=f00e97b9015d48bb96afd474bf86c1b8, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:37.684129 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81bc07-a6f5-49aa-bd2b-071b3bd7130c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:37.699499 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1395/2772] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:41:37 2022] PUT /identity/v3/projects/95a49c5324904115986e3755f5f4d781/users/f00e97b9015d48bb96afd474bf86c1b8/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:41:37.710139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9b0e8-54ef-478b-8e91-8c21616aac7b None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:37.710724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9b0e8-54ef-478b-8e91-8c21616aac7b None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:37.710724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9b0e8-54ef-478b-8e91-8c21616aac7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:37.740371 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d9f9b0e8-54ef-478b-8e91-8c21616aac7b None None] MFA Rules not processed for user `f00e97b9015d48bb96afd474bf86c1b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:37.741697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d9f9b0e8-54ef-478b-8e91-8c21616aac7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:37.761506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:37.810165 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1378/2773] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:41:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:37.814666 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81bc07-a6f5-49aa-bd2b-071b3bd7130c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:37.876779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81bc07-a6f5-49aa-bd2b-071b3bd7130c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:37.881996 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558953e-38a8-489b-8299-717d4c2bde55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:37.882610 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558953e-38a8-489b-8299-717d4c2bde55 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:37.882857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558953e-38a8-489b-8299-717d4c2bde55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:37.883287 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558953e-38a8-489b-8299-717d4c2bde55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:37.884373 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558953e-38a8-489b-8299-717d4c2bde55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:37.888132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558953e-38a8-489b-8299-717d4c2bde55 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:37.922415 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1396/2774] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:41:37 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:41:37.923708 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9b0e8-54ef-478b-8e91-8c21616aac7b None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:37.935018 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9b0e8-54ef-478b-8e91-8c21616aac7b None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:37.992189 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc50fac-46b9-47dc-b5bf-86811b145afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:37.992843 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc50fac-46b9-47dc-b5bf-86811b145afd admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:37.993138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc50fac-46b9-47dc-b5bf-86811b145afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:37.993404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc50fac-46b9-47dc-b5bf-86811b145afd admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:37.994029 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc50fac-46b9-47dc-b5bf-86811b145afd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:37.995949 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc50fac-46b9-47dc-b5bf-86811b145afd admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:38.019178 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1379/2775] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:41:37 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:41:38.023343 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558953e-38a8-489b-8299-717d4c2bde55 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:38.034356 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558953e-38a8-489b-8299-717d4c2bde55 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:38.041383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24ff45-2a66-405e-9cd7-db6b387b6a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:38.041981 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24ff45-2a66-405e-9cd7-db6b387b6a63 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:38.042262 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24ff45-2a66-405e-9cd7-db6b387b6a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:38.042533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24ff45-2a66-405e-9cd7-db6b387b6a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:38.043210 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24ff45-2a66-405e-9cd7-db6b387b6a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:38.044877 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24ff45-2a66-405e-9cd7-db6b387b6a63 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:38.056456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1397/2776] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:41:38.061806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc50fac-46b9-47dc-b5bf-86811b145afd admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:38.075353 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc50fac-46b9-47dc-b5bf-86811b145afd admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:38.081332 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06833b-100e-46ee-bfe0-c14824785af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:38.081984 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06833b-100e-46ee-bfe0-c14824785af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:38.082259 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06833b-100e-46ee-bfe0-c14824785af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:38.082539 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06833b-100e-46ee-bfe0-c14824785af9 admin admin] PATH_INFO: `/v3/projects/e29951ed124b4695b32658c0483e0834/users/e3fe1348eac24f1f8f7286a44331d9d7/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:38.100286 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06833b-100e-46ee-bfe0-c14824785af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29951ed124b4695b32658c0483e0834, user_id=e3fe1348eac24f1f8f7286a44331d9d7, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:38.103011 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06833b-100e-46ee-bfe0-c14824785af9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:38.118482 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1380/2777] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:41:38 2022] PUT /identity/v3/projects/e29951ed124b4695b32658c0483e0834/users/e3fe1348eac24f1f8f7286a44331d9d7/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:41:38.123506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24ff45-2a66-405e-9cd7-db6b387b6a63 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:38.136599 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24ff45-2a66-405e-9cd7-db6b387b6a63 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:38.207235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b63df2-5302-4f75-8285-f9b8e1de1270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:38.208170 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b63df2-5302-4f75-8285-f9b8e1de1270 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:38.208466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b63df2-5302-4f75-8285-f9b8e1de1270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:38.208760 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b63df2-5302-4f75-8285-f9b8e1de1270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:38.209404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b63df2-5302-4f75-8285-f9b8e1de1270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:38.211090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b63df2-5302-4f75-8285-f9b8e1de1270 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:38.221146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1398/2778] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:41:38.225692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06833b-100e-46ee-bfe0-c14824785af9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:38.236517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06833b-100e-46ee-bfe0-c14824785af9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:38.241943 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d585065-29b3-4986-94e0-85934b7a5889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:38.242594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d585065-29b3-4986-94e0-85934b7a5889 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:38.242873 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d585065-29b3-4986-94e0-85934b7a5889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:38.243175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d585065-29b3-4986-94e0-85934b7a5889 admin admin] PATH_INFO: `/v3/projects/e29951ed124b4695b32658c0483e0834/users/e3fe1348eac24f1f8f7286a44331d9d7/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:38.245255 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d585065-29b3-4986-94e0-85934b7a5889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29951ed124b4695b32658c0483e0834, user_id=e3fe1348eac24f1f8f7286a44331d9d7, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:38.247895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d585065-29b3-4986-94e0-85934b7a5889 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:38.259085 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1381/2779] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:41:38 2022] PUT /identity/v3/projects/e29951ed124b4695b32658c0483e0834/users/e3fe1348eac24f1f8f7286a44331d9d7/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:38.270364 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8056ab0-5966-4509-8543-cadedba4d412 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:38.270814 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8056ab0-5966-4509-8543-cadedba4d412 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:38.271456 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8056ab0-5966-4509-8543-cadedba4d412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:38.305920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-b8056ab0-5966-4509-8543-cadedba4d412 None None] MFA Rules not processed for user `e3fe1348eac24f1f8f7286a44331d9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:38.307147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-b8056ab0-5966-4509-8543-cadedba4d412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:38.345711 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:38.384712 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1399/2780] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:41:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:41:38.395841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d585065-29b3-4986-94e0-85934b7a5889 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:38.397635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4d585065-29b3-4986-94e0-85934b7a5889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:38.457762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d585065-29b3-4986-94e0-85934b7a5889 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:38.462225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c3c83-c3bc-40dc-a3f2-33d2c0f66cf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:38.462977 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c3c83-c3bc-40dc-a3f2-33d2c0f66cf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:38.463245 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c3c83-c3bc-40dc-a3f2-33d2c0f66cf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:38.463517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c3c83-c3bc-40dc-a3f2-33d2c0f66cf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:38.496971 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:38.531742 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c3c83-c3bc-40dc-a3f2-33d2c0f66cf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:38.533229 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c3c83-c3bc-40dc-a3f2-33d2c0f66cf4 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:38.543514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1382/2781] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:41:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:41:43.231984 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8056ab0-5966-4509-8543-cadedba4d412 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:43.243457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8056ab0-5966-4509-8543-cadedba4d412 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:43.247233 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108ce31-8134-4e6d-bc35-073bf7142307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:43.247839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108ce31-8134-4e6d-bc35-073bf7142307 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:43.248116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108ce31-8134-4e6d-bc35-073bf7142307 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:43.248385 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108ce31-8134-4e6d-bc35-073bf7142307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:43.258061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108ce31-8134-4e6d-bc35-073bf7142307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:43.259494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108ce31-8134-4e6d-bc35-073bf7142307 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:43.269042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1400/2782] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:41:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:41:48.539915 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=14 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:53.265128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=19 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:01.020436 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be15de-84dd-4bc2-9354-8c1749648544 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:01.020989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be15de-84dd-4bc2-9354-8c1749648544 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:01.021108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be15de-84dd-4bc2-9354-8c1749648544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:01.050970 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-d3be15de-84dd-4bc2-9354-8c1749648544 None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:01.052334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d3be15de-84dd-4bc2-9354-8c1749648544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:01.120160 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1383/2783] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:42:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:01.176021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108ce31-8134-4e6d-bc35-073bf7142307 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:01.177512 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-1108ce31-8134-4e6d-bc35-073bf7142307 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:01.253027 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108ce31-8134-4e6d-bc35-073bf7142307 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:01.259600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38c377-6ba3-4cdb-ad24-5dafbbfa495f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:01.260219 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38c377-6ba3-4cdb-ad24-5dafbbfa495f service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:01.260493 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38c377-6ba3-4cdb-ad24-5dafbbfa495f service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:01.260799 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38c377-6ba3-4cdb-ad24-5dafbbfa495f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:01.270689 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38c377-6ba3-4cdb-ad24-5dafbbfa495f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:01.272290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38c377-6ba3-4cdb-ad24-5dafbbfa495f service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:01.281178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1401/2784] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:42:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:42:09.659874 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be15de-84dd-4bc2-9354-8c1749648544 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:09.735406 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be15de-84dd-4bc2-9354-8c1749648544 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:09.742749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:09.743372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:09.744366 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:09.744785 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] PATH_INFO: `/v3/users/f00e97b9015d48bb96afd474bf86c1b8` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:09.746057 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00e97b9015d48bb96afd474bf86c1b8)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:09.748510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:09.804868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00e97b9015d48bb96afd474bf86c1b8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:09.805605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00e97b9015d48bb96afd474bf86c1b8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:09.806314 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00e97b9015d48bb96afd474bf86c1b8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:09.820723 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00e97b9015d48bb96afd474bf86c1b8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:09.829155 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00e97b9015d48bb96afd474bf86c1b8'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:09.843700 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1384/2785] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:42:09 2022] DELETE /identity/v3/users/f00e97b9015d48bb96afd474bf86c1b8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:09.848504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef0aba-5b94-426d-8499-6243eb296785 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:09.849140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef0aba-5b94-426d-8499-6243eb296785 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:09.849312 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef0aba-5b94-426d-8499-6243eb296785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:09.870195 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-53ef0aba-5b94-426d-8499-6243eb296785 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:09.871695 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-53ef0aba-5b94-426d-8499-6243eb296785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:09.945354 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1402/2786] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:09.956550 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:09.958830 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:09.984764 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:10.046144 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45f3ee-1be1-4f58-b473-8c4c180ca3a8 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:10.052427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87984e2-1721-4c65-a5d0-1738d6b481d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:10.053069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87984e2-1721-4c65-a5d0-1738d6b481d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:10.053342 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87984e2-1721-4c65-a5d0-1738d6b481d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:10.053607 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87984e2-1721-4c65-a5d0-1738d6b481d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:10.063459 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87984e2-1721-4c65-a5d0-1738d6b481d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:10.064932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87984e2-1721-4c65-a5d0-1738d6b481d2 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:10.080837 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1385/2787] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:10.387785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef0aba-5b94-426d-8499-6243eb296785 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:10.391460 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-53ef0aba-5b94-426d-8499-6243eb296785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:10.432885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef0aba-5b94-426d-8499-6243eb296785 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:10.442806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:10.443496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:10.443779 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:10.444050 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] PATH_INFO: `/v3/projects/95a49c5324904115986e3755f5f4d781` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:10.445263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] RBAC: Authorizing `identity:delete_project(project_id=95a49c5324904115986e3755f5f4d781)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:10.448724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:10.518284 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95a49c5324904115986e3755f5f4d781', 'request_id': 'req-672e7b6b-2d52-41b4-a6ea-c62188d8986f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:10.528412 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95a49c5324904115986e3755f5f4d781', 'request_id': 'req-672e7b6b-2d52-41b4-a6ea-c62188d8986f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:10.536941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95a49c5324904115986e3755f5f4d781', 'request_id': 'req-672e7b6b-2d52-41b4-a6ea-c62188d8986f'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:10.556741 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] The token cache is being invalidate because project 95a49c5324904115986e3755f5f4d781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:42:10.557338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:10.567324 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:10.574404 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1403/2788] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:42:10 2022] DELETE /identity/v3/projects/95a49c5324904115986e3755f5f4d781 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:10.575627 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87984e2-1721-4c65-a5d0-1738d6b481d2 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:10.586381 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87984e2-1721-4c65-a5d0-1738d6b481d2 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:10.679854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a208ec44-0e18-43c3-9a73-72d82ec16817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:10.680620 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208ec44-0e18-43c3-9a73-72d82ec16817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:10.680975 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208ec44-0e18-43c3-9a73-72d82ec16817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:10.681113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a208ec44-0e18-43c3-9a73-72d82ec16817 admin admin] PATH_INFO: `/v3/users/e3fe1348eac24f1f8f7286a44331d9d7` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:10.682282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208ec44-0e18-43c3-9a73-72d82ec16817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3fe1348eac24f1f8f7286a44331d9d7)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:10.684351 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a208ec44-0e18-43c3-9a73-72d82ec16817 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:10.746139 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a208ec44-0e18-43c3-9a73-72d82ec16817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3fe1348eac24f1f8f7286a44331d9d7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:10.746900 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a208ec44-0e18-43c3-9a73-72d82ec16817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3fe1348eac24f1f8f7286a44331d9d7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:10.747508 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a208ec44-0e18-43c3-9a73-72d82ec16817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3fe1348eac24f1f8f7286a44331d9d7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:10.768614 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a208ec44-0e18-43c3-9a73-72d82ec16817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3fe1348eac24f1f8f7286a44331d9d7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:10.781345 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-a208ec44-0e18-43c3-9a73-72d82ec16817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3fe1348eac24f1f8f7286a44331d9d7'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:10.799621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1386/2789] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:42:10 2022] DELETE /identity/v3/users/e3fe1348eac24f1f8f7286a44331d9d7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:42:11.155598 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:11.157576 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:11.176839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e7b6b-2d52-41b4-a6ea-c62188d8986f admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:11.195920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:11.269159 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c643a6-72d8-4059-beb9-661fef03d389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:11.270438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c643a6-72d8-4059-beb9-661fef03d389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:11.270985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c643a6-72d8-4059-beb9-661fef03d389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:11.271402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c643a6-72d8-4059-beb9-661fef03d389 admin admin] PATH_INFO: `/v3/projects/e29951ed124b4695b32658c0483e0834` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:11.272822 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c643a6-72d8-4059-beb9-661fef03d389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e29951ed124b4695b32658c0483e0834)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:11.274905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c643a6-72d8-4059-beb9-661fef03d389 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:11.337764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b8c643a6-72d8-4059-beb9-661fef03d389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e29951ed124b4695b32658c0483e0834', 'request_id': 'req-b8c643a6-72d8-4059-beb9-661fef03d389'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:11.350101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b8c643a6-72d8-4059-beb9-661fef03d389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e29951ed124b4695b32658c0483e0834', 'request_id': 'req-b8c643a6-72d8-4059-beb9-661fef03d389'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:11.357786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b8c643a6-72d8-4059-beb9-661fef03d389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e29951ed124b4695b32658c0483e0834', 'request_id': 'req-b8c643a6-72d8-4059-beb9-661fef03d389'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:11.373823 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b8c643a6-72d8-4059-beb9-661fef03d389 admin admin] The token cache is being invalidate because project e29951ed124b4695b32658c0483e0834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:42:11.374083 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b8c643a6-72d8-4059-beb9-661fef03d389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:11.374778 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-b8c643a6-72d8-4059-beb9-661fef03d389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:11.376520 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1404/2790] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:42:11 2022] DELETE /identity/v3/projects/e29951ed124b4695b32658c0483e0834 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:11.383497 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5e720-9a2b-4142-b46d-c552e8a8c704 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:11.384084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5e720-9a2b-4142-b46d-c552e8a8c704 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:11.384566 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5e720-9a2b-4142-b46d-c552e8a8c704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:11.408208 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-4cd5e720-9a2b-4142-b46d-c552e8a8c704 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:11.410270 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-4cd5e720-9a2b-4142-b46d-c552e8a8c704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:11.439792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:11.496255 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1387/2791] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:11.506186 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1c72e-6010-4f9e-b6e0-d70bbf973154 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:11.506494 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1c72e-6010-4f9e-b6e0-d70bbf973154 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:11.506988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1c72e-6010-4f9e-b6e0-d70bbf973154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:11.528038 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-13c1c72e-6010-4f9e-b6e0-d70bbf973154 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:11.529310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-13c1c72e-6010-4f9e-b6e0-d70bbf973154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:11.540007 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1405/2792] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:11.543890 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5e720-9a2b-4142-b46d-c552e8a8c704 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:11.555505 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5e720-9a2b-4142-b46d-c552e8a8c704 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:11.561858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f093ca-9edc-4551-9dbf-b19e672703c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:11.562491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f093ca-9edc-4551-9dbf-b19e672703c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:11.562768 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f093ca-9edc-4551-9dbf-b19e672703c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:11.563034 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f093ca-9edc-4551-9dbf-b19e672703c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:11.563718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f093ca-9edc-4551-9dbf-b19e672703c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:42:11.564033 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f093ca-9edc-4551-9dbf-b19e672703c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:11.566061 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f093ca-9edc-4551-9dbf-b19e672703c5 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:11.580697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1388/2793] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:42:11.584827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1c72e-6010-4f9e-b6e0-d70bbf973154 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:11.595863 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1c72e-6010-4f9e-b6e0-d70bbf973154 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:11.602248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090f57a-8d22-435a-b69c-8bbdb9bc8b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:11.602975 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090f57a-8d22-435a-b69c-8bbdb9bc8b5a admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:11.603315 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090f57a-8d22-435a-b69c-8bbdb9bc8b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:11.603652 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090f57a-8d22-435a-b69c-8bbdb9bc8b5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:11.604369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090f57a-8d22-435a-b69c-8bbdb9bc8b5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:11.606410 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090f57a-8d22-435a-b69c-8bbdb9bc8b5a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:11.641742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1406/2794] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:42:11 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:42:11.645967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f093ca-9edc-4551-9dbf-b19e672703c5 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:11.656605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f093ca-9edc-4551-9dbf-b19e672703c5 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:11.719048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780e6b2-0ee7-49d7-bbd1-f7bed742b4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:11.719798 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780e6b2-0ee7-49d7-bbd1-f7bed742b4cf admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:11.719913 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780e6b2-0ee7-49d7-bbd1-f7bed742b4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:11.720173 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780e6b2-0ee7-49d7-bbd1-f7bed742b4cf admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:11.723133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780e6b2-0ee7-49d7-bbd1-f7bed742b4cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:11.723133 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780e6b2-0ee7-49d7-bbd1-f7bed742b4cf admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:11.750197 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1389/2795] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:42:11 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:42:11.754382 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090f57a-8d22-435a-b69c-8bbdb9bc8b5a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:11.766574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090f57a-8d22-435a-b69c-8bbdb9bc8b5a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:11.771784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc8421-ff8f-496c-aae0-19b492ecb67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:11.772457 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc8421-ff8f-496c-aae0-19b492ecb67d admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:11.772845 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc8421-ff8f-496c-aae0-19b492ecb67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:11.773327 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc8421-ff8f-496c-aae0-19b492ecb67d admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:11.774047 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc8421-ff8f-496c-aae0-19b492ecb67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:11.775747 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc8421-ff8f-496c-aae0-19b492ecb67d admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:11.788857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1407/2796] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:42:11.792784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780e6b2-0ee7-49d7-bbd1-f7bed742b4cf admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:11.805646 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780e6b2-0ee7-49d7-bbd1-f7bed742b4cf admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:11.811549 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf2ce1-3084-4864-a339-26aea5bd2a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:11.812358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf2ce1-3084-4864-a339-26aea5bd2a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:11.812854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf2ce1-3084-4864-a339-26aea5bd2a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:11.813122 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf2ce1-3084-4864-a339-26aea5bd2a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:11.813766 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf2ce1-3084-4864-a339-26aea5bd2a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:11.815537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf2ce1-3084-4864-a339-26aea5bd2a10 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:11.828465 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1390/2797] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:42:11.832759 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc8421-ff8f-496c-aae0-19b492ecb67d admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:11.845093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc8421-ff8f-496c-aae0-19b492ecb67d admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:11.850941 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b2890-838d-44b3-bf66-99035a742b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:11.851555 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b2890-838d-44b3-bf66-99035a742b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:11.851836 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b2890-838d-44b3-bf66-99035a742b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:11.852111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b2890-838d-44b3-bf66-99035a742b00 admin admin] PATH_INFO: `/v3/projects/14e0fcad39ca4275a418235182b41cbc/users/bf821f97c3bd4047b9ceed196616147a/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:11.869905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b2890-838d-44b3-bf66-99035a742b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14e0fcad39ca4275a418235182b41cbc, user_id=bf821f97c3bd4047b9ceed196616147a, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:11.872967 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b2890-838d-44b3-bf66-99035a742b00 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:11.896706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1408/2798] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:42:11 2022] PUT /identity/v3/projects/14e0fcad39ca4275a418235182b41cbc/users/bf821f97c3bd4047b9ceed196616147a/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:42:11.960596 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5e8eb-ace7-46c8-a994-c02de4a3bb0a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:11.961046 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5e8eb-ace7-46c8-a994-c02de4a3bb0a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:11.961417 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5e8eb-ace7-46c8-a994-c02de4a3bb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:11.994435 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6ac5e8eb-ace7-46c8-a994-c02de4a3bb0a None None] MFA Rules not processed for user `bf821f97c3bd4047b9ceed196616147a`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:11.995938 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6ac5e8eb-ace7-46c8-a994-c02de4a3bb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:12.076688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1391/2799] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:42:12.089833 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b2890-838d-44b3-bf66-99035a742b00 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:12.091896 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5d2b2890-838d-44b3-bf66-99035a742b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:12.152029 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:12.170942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b2890-838d-44b3-bf66-99035a742b00 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:12.175066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d2ff3-ee5f-4b9c-b20a-e81f6f198404 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94bfed415d24a9cb596b1fa92d5df97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:12.175762 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d2ff3-ee5f-4b9c-b20a-e81f6f198404 service glance] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:12.176037 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d2ff3-ee5f-4b9c-b20a-e81f6f198404 service glance] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:12.176429 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d2ff3-ee5f-4b9c-b20a-e81f6f198404 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:12.186347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d2ff3-ee5f-4b9c-b20a-e81f6f198404 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:12.187823 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d2ff3-ee5f-4b9c-b20a-e81f6f198404 service glance] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:12.198930 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1409/2800] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:18.641394 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6ec8-26c3-4534-862e-46040df0f767 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:18.641825 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6ec8-26c3-4534-862e-46040df0f767 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:18.641922 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6ec8-26c3-4534-862e-46040df0f767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:18.665322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-0e5e6ec8-26c3-4534-862e-46040df0f767 None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:18.667446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0e5e6ec8-26c3-4534-862e-46040df0f767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:18.701906 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:18.743200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1392/2801] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 24 12:42:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:18.756092 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d2ff3-ee5f-4b9c-b20a-e81f6f198404 service glance] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:18.757585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a30d2ff3-ee5f-4b9c-b20a-e81f6f198404 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:18.830688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d2ff3-ee5f-4b9c-b20a-e81f6f198404 service glance] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:18.837094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f176e1-636e-4aa5-bb90-83a352bfdd2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:18.837709 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f176e1-636e-4aa5-bb90-83a352bfdd2c service swift] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:18.837987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f176e1-636e-4aa5-bb90-83a352bfdd2c service swift] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:18.838240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f176e1-636e-4aa5-bb90-83a352bfdd2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:18.857141 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f176e1-636e-4aa5-bb90-83a352bfdd2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:18.858878 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f176e1-636e-4aa5-bb90-83a352bfdd2c service swift] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:18.869718 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1410/2802] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:42:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:42:20.902676 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e6ec8-26c3-4534-862e-46040df0f767 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:20.904168 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0e5e6ec8-26c3-4534-862e-46040df0f767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:20.980894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e6ec8-26c3-4534-862e-46040df0f767 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:20.985858 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a819de-1f0d-4881-ace0-13becdb73180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:20.986521 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a819de-1f0d-4881-ace0-13becdb73180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:20.986810 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a819de-1f0d-4881-ace0-13becdb73180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:20.987084 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a819de-1f0d-4881-ace0-13becdb73180 admin admin] PATH_INFO: `/v3/users/85620ae3c59a44da898ec76436271027` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:20.988389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a819de-1f0d-4881-ace0-13becdb73180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85620ae3c59a44da898ec76436271027)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:20.990897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a819de-1f0d-4881-ace0-13becdb73180 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:21.044845 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f6a819de-1f0d-4881-ace0-13becdb73180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85620ae3c59a44da898ec76436271027'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:21.045586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f6a819de-1f0d-4881-ace0-13becdb73180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85620ae3c59a44da898ec76436271027'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:21.046298 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f6a819de-1f0d-4881-ace0-13becdb73180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85620ae3c59a44da898ec76436271027'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:21.059443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f6a819de-1f0d-4881-ace0-13becdb73180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85620ae3c59a44da898ec76436271027'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:21.068797 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-f6a819de-1f0d-4881-ace0-13becdb73180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85620ae3c59a44da898ec76436271027'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:21.083654 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1393/2803] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:42:20 2022] DELETE /identity/v3/users/85620ae3c59a44da898ec76436271027 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:21.285873 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f176e1-636e-4aa5-bb90-83a352bfdd2c service swift] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:21.297432 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f176e1-636e-4aa5-bb90-83a352bfdd2c service swift] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:21.360762 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9600fe5-3735-448a-b8a3-10b5bad64fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:21.361402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9600fe5-3735-448a-b8a3-10b5bad64fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:21.361682 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9600fe5-3735-448a-b8a3-10b5bad64fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:21.361953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9600fe5-3735-448a-b8a3-10b5bad64fe8 admin admin] PATH_INFO: `/v3/projects/121d2511f42847cab3930da359a3a7ff` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:21.363095 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9600fe5-3735-448a-b8a3-10b5bad64fe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=121d2511f42847cab3930da359a3a7ff)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:21.365252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9600fe5-3735-448a-b8a3-10b5bad64fe8 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:21.431094 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9600fe5-3735-448a-b8a3-10b5bad64fe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '121d2511f42847cab3930da359a3a7ff', 'request_id': 'req-a9600fe5-3735-448a-b8a3-10b5bad64fe8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:21.440060 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9600fe5-3735-448a-b8a3-10b5bad64fe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '121d2511f42847cab3930da359a3a7ff', 'request_id': 'req-a9600fe5-3735-448a-b8a3-10b5bad64fe8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:21.446656 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9600fe5-3735-448a-b8a3-10b5bad64fe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '121d2511f42847cab3930da359a3a7ff', 'request_id': 'req-a9600fe5-3735-448a-b8a3-10b5bad64fe8'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:21.458248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9600fe5-3735-448a-b8a3-10b5bad64fe8 admin admin] The token cache is being invalidate because project 121d2511f42847cab3930da359a3a7ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:42:21.458533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9600fe5-3735-448a-b8a3-10b5bad64fe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:21.459361 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-a9600fe5-3735-448a-b8a3-10b5bad64fe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:21.461558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1411/2804] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:42:21 2022] DELETE /identity/v3/projects/121d2511f42847cab3930da359a3a7ff => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:21.467777 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5934169-4870-4b6e-bb6d-a31237b56bae None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:21.468068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5934169-4870-4b6e-bb6d-a31237b56bae None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:21.468335 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5934169-4870-4b6e-bb6d-a31237b56bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:21.488579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-a5934169-4870-4b6e-bb6d-a31237b56bae None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:21.489908 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a5934169-4870-4b6e-bb6d-a31237b56bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:21.546909 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:21.564317 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1394/2805] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:21.572254 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2a74d-00f2-4458-94d8-77db5785350e None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:21.572537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2a74d-00f2-4458-94d8-77db5785350e None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:21.572840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2a74d-00f2-4458-94d8-77db5785350e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:21.580751 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:21.593926 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-aae2a74d-00f2-4458-94d8-77db5785350e None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:21.595713 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-aae2a74d-00f2-4458-94d8-77db5785350e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:21.603740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1412/2806] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:21.608082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5934169-4870-4b6e-bb6d-a31237b56bae None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:21.618725 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5934169-4870-4b6e-bb6d-a31237b56bae None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:21.629000 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19fba6-a742-4190-9c8c-ee8d9c52c044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:21.629650 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19fba6-a742-4190-9c8c-ee8d9c52c044 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:21.630385 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19fba6-a742-4190-9c8c-ee8d9c52c044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:21.630673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19fba6-a742-4190-9c8c-ee8d9c52c044 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:21.631422 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19fba6-a742-4190-9c8c-ee8d9c52c044 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:42:21.631758 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19fba6-a742-4190-9c8c-ee8d9c52c044 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:21.633393 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19fba6-a742-4190-9c8c-ee8d9c52c044 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:21.646236 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1395/2807] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:42:21.650958 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2a74d-00f2-4458-94d8-77db5785350e None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:21.663635 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2a74d-00f2-4458-94d8-77db5785350e None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:21.669661 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed218c2-42fb-4a10-8238-3a857c12ec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:21.670263 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed218c2-42fb-4a10-8238-3a857c12ec50 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:21.670543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed218c2-42fb-4a10-8238-3a857c12ec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:21.670809 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed218c2-42fb-4a10-8238-3a857c12ec50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:21.671430 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed218c2-42fb-4a10-8238-3a857c12ec50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:21.673407 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed218c2-42fb-4a10-8238-3a857c12ec50 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:21.704793 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1413/2808] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:42:21 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:42:21.709557 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19fba6-a742-4190-9c8c-ee8d9c52c044 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:21.725460 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19fba6-a742-4190-9c8c-ee8d9c52c044 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:21.784433 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac832906-26ce-442b-ac29-6386e2a76ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:21.785097 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac832906-26ce-442b-ac29-6386e2a76ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:21.785334 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac832906-26ce-442b-ac29-6386e2a76ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:21.785598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac832906-26ce-442b-ac29-6386e2a76ee2 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:21.786531 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac832906-26ce-442b-ac29-6386e2a76ee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:21.788662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac832906-26ce-442b-ac29-6386e2a76ee2 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:21.812932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1396/2809] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:42:21 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:42:21.817602 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed218c2-42fb-4a10-8238-3a857c12ec50 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:21.832255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed218c2-42fb-4a10-8238-3a857c12ec50 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:21.838362 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f62caa-334d-4263-b4ea-8a06bf77f5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:21.838972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f62caa-334d-4263-b4ea-8a06bf77f5ba admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:21.839246 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f62caa-334d-4263-b4ea-8a06bf77f5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:21.839552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f62caa-334d-4263-b4ea-8a06bf77f5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:21.840200 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f62caa-334d-4263-b4ea-8a06bf77f5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:21.841825 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f62caa-334d-4263-b4ea-8a06bf77f5ba admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:21.852827 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1414/2810] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:42:21.857099 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac832906-26ce-442b-ac29-6386e2a76ee2 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:21.867506 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac832906-26ce-442b-ac29-6386e2a76ee2 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:21.872594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be03821-8934-4809-8179-53abd6ff1f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:21.873170 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be03821-8934-4809-8179-53abd6ff1f48 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:21.873427 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be03821-8934-4809-8179-53abd6ff1f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:21.873693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be03821-8934-4809-8179-53abd6ff1f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:21.874323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be03821-8934-4809-8179-53abd6ff1f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:21.875924 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be03821-8934-4809-8179-53abd6ff1f48 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:21.896068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1397/2811] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:42:21.901690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f62caa-334d-4263-b4ea-8a06bf77f5ba admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:21.916590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f62caa-334d-4263-b4ea-8a06bf77f5ba admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:21.922190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79a2ea-3247-49b9-b0b9-2ac00bb317fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:21.922831 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79a2ea-3247-49b9-b0b9-2ac00bb317fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:21.923107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79a2ea-3247-49b9-b0b9-2ac00bb317fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:21.923378 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79a2ea-3247-49b9-b0b9-2ac00bb317fb admin admin] PATH_INFO: `/v3/projects/387a66095d4c444ab9be10d6f42792a0/users/644126953b3144cebf35d0ec051db03a/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:21.940437 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79a2ea-3247-49b9-b0b9-2ac00bb317fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=387a66095d4c444ab9be10d6f42792a0, user_id=644126953b3144cebf35d0ec051db03a, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:21.943290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79a2ea-3247-49b9-b0b9-2ac00bb317fb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:21.962725 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1415/2812] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:42:21 2022] PUT /identity/v3/projects/387a66095d4c444ab9be10d6f42792a0/users/644126953b3144cebf35d0ec051db03a/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:21.974200 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bdba4-e6da-4fb7-9b67-934eb12ed08a None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:21.974743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bdba4-e6da-4fb7-9b67-934eb12ed08a None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:21.974853 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bdba4-e6da-4fb7-9b67-934eb12ed08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:22.003048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-c54bdba4-e6da-4fb7-9b67-934eb12ed08a None None] MFA Rules not processed for user `644126953b3144cebf35d0ec051db03a`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:22.004271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c54bdba4-e6da-4fb7-9b67-934eb12ed08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:22.075096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1398/2813] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:42:22.086884 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79a2ea-3247-49b9-b0b9-2ac00bb317fb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:22.088901 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5d79a2ea-3247-49b9-b0b9-2ac00bb317fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:22.149942 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79a2ea-3247-49b9-b0b9-2ac00bb317fb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:22.154020 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474a6fb-0540-4b77-b117-b05a63bb6f29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:22.154621 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474a6fb-0540-4b77-b117-b05a63bb6f29 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:22.154894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474a6fb-0540-4b77-b117-b05a63bb6f29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:22.155157 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474a6fb-0540-4b77-b117-b05a63bb6f29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:22.164715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474a6fb-0540-4b77-b117-b05a63bb6f29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:22.166205 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474a6fb-0540-4b77-b117-b05a63bb6f29 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:22.176471 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1416/2814] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:32.071495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:32.172526 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:38.530989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bdba4-e6da-4fb7-9b67-934eb12ed08a None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:38.533094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c54bdba4-e6da-4fb7-9b67-934eb12ed08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:38.606822 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bdba4-e6da-4fb7-9b67-934eb12ed08a None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:38.612172 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:38.612989 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:38.613265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:38.613538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] PATH_INFO: `/v3/users/bf821f97c3bd4047b9ceed196616147a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:38.614662 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf821f97c3bd4047b9ceed196616147a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:38.617180 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:38.669814 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf821f97c3bd4047b9ceed196616147a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:38.670485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf821f97c3bd4047b9ceed196616147a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:38.671102 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf821f97c3bd4047b9ceed196616147a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:38.683203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf821f97c3bd4047b9ceed196616147a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:38.692028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf821f97c3bd4047b9ceed196616147a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:38.704736 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1399/2815] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:42:38 2022] DELETE /identity/v3/users/bf821f97c3bd4047b9ceed196616147a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:38.709998 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505643e9-830a-4952-b17e-101ecdde29d2 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:38.709998 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505643e9-830a-4952-b17e-101ecdde29d2 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:38.710587 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505643e9-830a-4952-b17e-101ecdde29d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:38.730534 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-505643e9-830a-4952-b17e-101ecdde29d2 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:38.731815 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-505643e9-830a-4952-b17e-101ecdde29d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:38.798938 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1417/2816] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:38.809490 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:38.811047 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:38.871295 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada90460-0887-4e14-af44-4c3ff1ff6600 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:38.875317 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2560340-7113-4a8c-b3eb-00f765b5ef98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:38.875916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2560340-7113-4a8c-b3eb-00f765b5ef98 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:38.876191 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2560340-7113-4a8c-b3eb-00f765b5ef98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:38.876455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2560340-7113-4a8c-b3eb-00f765b5ef98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:38.888839 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2560340-7113-4a8c-b3eb-00f765b5ef98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:38.890959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2560340-7113-4a8c-b3eb-00f765b5ef98 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:38.902138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1400/2817] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:39.198868 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505643e9-830a-4952-b17e-101ecdde29d2 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:39.200341 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-505643e9-830a-4952-b17e-101ecdde29d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:39.213647 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505643e9-830a-4952-b17e-101ecdde29d2 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:39.218464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645b594-4063-47ea-9ff1-b2d1daf4adb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:39.219105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645b594-4063-47ea-9ff1-b2d1daf4adb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:39.219402 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645b594-4063-47ea-9ff1-b2d1daf4adb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:39.219670 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645b594-4063-47ea-9ff1-b2d1daf4adb7 admin admin] PATH_INFO: `/v3/projects/14e0fcad39ca4275a418235182b41cbc` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:39.220835 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645b594-4063-47ea-9ff1-b2d1daf4adb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14e0fcad39ca4275a418235182b41cbc)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:39.222780 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645b594-4063-47ea-9ff1-b2d1daf4adb7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:39.282740 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3645b594-4063-47ea-9ff1-b2d1daf4adb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14e0fcad39ca4275a418235182b41cbc', 'request_id': 'req-3645b594-4063-47ea-9ff1-b2d1daf4adb7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:39.293115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3645b594-4063-47ea-9ff1-b2d1daf4adb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14e0fcad39ca4275a418235182b41cbc', 'request_id': 'req-3645b594-4063-47ea-9ff1-b2d1daf4adb7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:39.302885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3645b594-4063-47ea-9ff1-b2d1daf4adb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14e0fcad39ca4275a418235182b41cbc', 'request_id': 'req-3645b594-4063-47ea-9ff1-b2d1daf4adb7'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:39.313782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3645b594-4063-47ea-9ff1-b2d1daf4adb7 admin admin] The token cache is being invalidate because project 14e0fcad39ca4275a418235182b41cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:42:39.314068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3645b594-4063-47ea-9ff1-b2d1daf4adb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:39.314965 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-3645b594-4063-47ea-9ff1-b2d1daf4adb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:39.316617 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1418/2818] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:42:39 2022] DELETE /identity/v3/projects/14e0fcad39ca4275a418235182b41cbc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:39.321792 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a300414-e347-4b74-8af0-101d0317fed5 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:39.322075 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a300414-e347-4b74-8af0-101d0317fed5 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:39.322346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a300414-e347-4b74-8af0-101d0317fed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:39.342546 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-5a300414-e347-4b74-8af0-101d0317fed5 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:39.343827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5a300414-e347-4b74-8af0-101d0317fed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:39.361279 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:39.409888 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1401/2819] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:39.418248 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77908d12-5862-4f04-8e55-2add5998bd67 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:39.418535 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77908d12-5862-4f04-8e55-2add5998bd67 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:39.418808 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77908d12-5862-4f04-8e55-2add5998bd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:39.439132 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-77908d12-5862-4f04-8e55-2add5998bd67 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:39.440329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-77908d12-5862-4f04-8e55-2add5998bd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:39.448719 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1419/2820] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:39.453022 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a300414-e347-4b74-8af0-101d0317fed5 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:39.465012 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a300414-e347-4b74-8af0-101d0317fed5 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:39.470806 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02becae8-82b5-4bc3-be29-47e03a513bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:39.471390 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02becae8-82b5-4bc3-be29-47e03a513bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:39.471697 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02becae8-82b5-4bc3-be29-47e03a513bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:39.471964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02becae8-82b5-4bc3-be29-47e03a513bb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:39.472619 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02becae8-82b5-4bc3-be29-47e03a513bb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:42:39.472955 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02becae8-82b5-4bc3-be29-47e03a513bb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:39.474605 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02becae8-82b5-4bc3-be29-47e03a513bb1 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:39.488549 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1402/2821] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:42:39.493088 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77908d12-5862-4f04-8e55-2add5998bd67 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:39.502904 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77908d12-5862-4f04-8e55-2add5998bd67 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:39.507960 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e99048-8124-4f83-8d57-8e6778745c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:39.508562 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e99048-8124-4f83-8d57-8e6778745c2b admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:39.508875 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e99048-8124-4f83-8d57-8e6778745c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:39.509143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e99048-8124-4f83-8d57-8e6778745c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:39.509765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e99048-8124-4f83-8d57-8e6778745c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:39.511699 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e99048-8124-4f83-8d57-8e6778745c2b admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:39.553535 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1420/2822] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:42:39 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:42:39.557801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02becae8-82b5-4bc3-be29-47e03a513bb1 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:39.569116 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02becae8-82b5-4bc3-be29-47e03a513bb1 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:39.639734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e487c-ef1e-4463-a1c8-764b47341d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:39.640791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e487c-ef1e-4463-a1c8-764b47341d57 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:39.641119 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e487c-ef1e-4463-a1c8-764b47341d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:39.641396 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e487c-ef1e-4463-a1c8-764b47341d57 admin admin] PATH_INFO: `/v3/users` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:39.642037 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e487c-ef1e-4463-a1c8-764b47341d57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:39.643995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e487c-ef1e-4463-a1c8-764b47341d57 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:39.667074 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1403/2823] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:42:39 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:42:39.671660 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e99048-8124-4f83-8d57-8e6778745c2b admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:39.682789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e99048-8124-4f83-8d57-8e6778745c2b admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:39.687840 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5accbd1-e4e7-4c18-b56f-027114170a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:39.688470 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5accbd1-e4e7-4c18-b56f-027114170a37 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:39.688764 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5accbd1-e4e7-4c18-b56f-027114170a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:39.689042 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5accbd1-e4e7-4c18-b56f-027114170a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:39.689693 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5accbd1-e4e7-4c18-b56f-027114170a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:39.691338 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5accbd1-e4e7-4c18-b56f-027114170a37 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:39.702501 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1421/2824] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:42:39.707025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e487c-ef1e-4463-a1c8-764b47341d57 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:39.717887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e487c-ef1e-4463-a1c8-764b47341d57 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:39.722776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6f43d-ba5c-4993-926b-55ed6fced548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:39.723424 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6f43d-ba5c-4993-926b-55ed6fced548 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:39.723701 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6f43d-ba5c-4993-926b-55ed6fced548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:39.723940 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6f43d-ba5c-4993-926b-55ed6fced548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:39.724634 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6f43d-ba5c-4993-926b-55ed6fced548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:39.726477 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6f43d-ba5c-4993-926b-55ed6fced548 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:39.737744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1404/2825] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:42:39.742295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5accbd1-e4e7-4c18-b56f-027114170a37 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:39.753533 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5accbd1-e4e7-4c18-b56f-027114170a37 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:39.761107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee9b18-110c-4b13-ad7e-59c14f4770bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:39.761916 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee9b18-110c-4b13-ad7e-59c14f4770bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:39.762395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee9b18-110c-4b13-ad7e-59c14f4770bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:39.762684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee9b18-110c-4b13-ad7e-59c14f4770bb admin admin] PATH_INFO: `/v3/projects/c5da2f4e102b49438b2b80ba75d1bf0c/users/7159c045925f48e8a03594a7d38e9d27/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:39.778511 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee9b18-110c-4b13-ad7e-59c14f4770bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5da2f4e102b49438b2b80ba75d1bf0c, user_id=7159c045925f48e8a03594a7d38e9d27, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:39.781426 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee9b18-110c-4b13-ad7e-59c14f4770bb admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:39.799514 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1422/2826] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:42:39 2022] PUT /identity/v3/projects/c5da2f4e102b49438b2b80ba75d1bf0c/users/7159c045925f48e8a03594a7d38e9d27/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:39.816960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128d277-c508-47d8-9fa2-fad9c5237add None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:39.817213 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128d277-c508-47d8-9fa2-fad9c5237add None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:39.817502 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128d277-c508-47d8-9fa2-fad9c5237add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:39.854866 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-e128d277-c508-47d8-9fa2-fad9c5237add None None] MFA Rules not processed for user `7159c045925f48e8a03594a7d38e9d27`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:39.856096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-e128d277-c508-47d8-9fa2-fad9c5237add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:39.937518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1405/2827] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:39.952126 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee9b18-110c-4b13-ad7e-59c14f4770bb admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:39.954082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a4ee9b18-110c-4b13-ad7e-59c14f4770bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:39.968676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:40.021913 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee9b18-110c-4b13-ad7e-59c14f4770bb admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:40.026466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6aef9-3768-431a-bbc7-18c26e18db00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:40.027111 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6aef9-3768-431a-bbc7-18c26e18db00 service cinder] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:40.027396 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6aef9-3768-431a-bbc7-18c26e18db00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:40.027646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6aef9-3768-431a-bbc7-18c26e18db00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:40.036906 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6aef9-3768-431a-bbc7-18c26e18db00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:40.038585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6aef9-3768-431a-bbc7-18c26e18db00 service cinder] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:40.047514 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1423/2828] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:41.625124 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a591982-d9c0-4f97-9535-73fe8e497ec6 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:41.625671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a591982-d9c0-4f97-9535-73fe8e497ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:41.625671 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a591982-d9c0-4f97-9535-73fe8e497ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:41.645627 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-3a591982-d9c0-4f97-9535-73fe8e497ec6 None None] MFA Rules not processed for user `7159c045925f48e8a03594a7d38e9d27`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:41.646827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3a591982-d9c0-4f97-9535-73fe8e497ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:41.653763 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1406/2829] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:41.691022 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6aef9-3768-431a-bbc7-18c26e18db00 service cinder] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:41.692669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-20d6aef9-3768-431a-bbc7-18c26e18db00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:41.722278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a591982-d9c0-4f97-9535-73fe8e497ec6 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:41.723885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-3a591982-d9c0-4f97-9535-73fe8e497ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:41.737937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:41.755757 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6aef9-3768-431a-bbc7-18c26e18db00 service cinder] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:41.760558 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410ba71-89e8-4662-a5cf-4cfe8ff12844 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94bfed415d24a9cb596b1fa92d5df97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:41.760985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410ba71-89e8-4662-a5cf-4cfe8ff12844 service glance] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:41.761259 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410ba71-89e8-4662-a5cf-4cfe8ff12844 service glance] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:41.761537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410ba71-89e8-4662-a5cf-4cfe8ff12844 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:41.763351 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d410ba71-89e8-4662-a5cf-4cfe8ff12844 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:41.795930 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a591982-d9c0-4f97-9535-73fe8e497ec6 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:41.800467 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:41.801138 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:41.801407 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:41.801670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] PATH_INFO: `/v3/users/644126953b3144cebf35d0ec051db03a` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:41.802807 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=644126953b3144cebf35d0ec051db03a)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:41.804801 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:41.858044 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644126953b3144cebf35d0ec051db03a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:41.858676 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410ba71-89e8-4662-a5cf-4cfe8ff12844 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:41.858877 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644126953b3144cebf35d0ec051db03a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:41.859522 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644126953b3144cebf35d0ec051db03a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:41.860255 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410ba71-89e8-4662-a5cf-4cfe8ff12844 service glance] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:41.872514 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644126953b3144cebf35d0ec051db03a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:41.881276 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644126953b3144cebf35d0ec051db03a'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:41.881947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1424/2830] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:41.894271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1407/2831] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:42:41 2022] DELETE /identity/v3/users/644126953b3144cebf35d0ec051db03a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:41.898819 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831bb78-869a-4b3e-9b93-7de7f58b31bf None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:41.899101 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831bb78-869a-4b3e-9b93-7de7f58b31bf None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:41.899369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831bb78-869a-4b3e-9b93-7de7f58b31bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:41.920110 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d831bb78-869a-4b3e-9b93-7de7f58b31bf None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:41.921329 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d831bb78-869a-4b3e-9b93-7de7f58b31bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:41.954313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:41.988067 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1425/2832] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:41.998175 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:41.999677 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:42.058394 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ce068-3cb7-4f92-aa6d-1fe43b51ae1a admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:42.062063 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fc03f-02f7-4714-b798-5bda52d1e02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:42.062660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fc03f-02f7-4714-b798-5bda52d1e02d service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:42.062935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fc03f-02f7-4714-b798-5bda52d1e02d service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:42.063202 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fc03f-02f7-4714-b798-5bda52d1e02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:42.073095 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fc03f-02f7-4714-b798-5bda52d1e02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:42.075096 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fc03f-02f7-4714-b798-5bda52d1e02d service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:42.086778 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1408/2833] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:42.356289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831bb78-869a-4b3e-9b93-7de7f58b31bf None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:42.359988 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d831bb78-869a-4b3e-9b93-7de7f58b31bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:42.372284 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831bb78-869a-4b3e-9b93-7de7f58b31bf None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:42.385581 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fc03f-02f7-4714-b798-5bda52d1e02d service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:42.387080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c35fc03f-02f7-4714-b798-5bda52d1e02d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:42.388140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73c8d4-f82f-45e7-8fcb-64038013c710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:42.389116 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73c8d4-f82f-45e7-8fcb-64038013c710 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:42.389653 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73c8d4-f82f-45e7-8fcb-64038013c710 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:42.390143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73c8d4-f82f-45e7-8fcb-64038013c710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:42.399257 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-cf73c8d4-f82f-45e7-8fcb-64038013c710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:42.428174 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fc03f-02f7-4714-b798-5bda52d1e02d service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:42.434525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:42.435597 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:42.435878 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:42.436148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19 admin admin] PATH_INFO: `/v3/projects/387a66095d4c444ab9be10d6f42792a0` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:42.437476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=387a66095d4c444ab9be10d6f42792a0)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:42.439548 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:42.489833 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:42.502580 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73c8d4-f82f-45e7-8fcb-64038013c710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:42.504269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73c8d4-f82f-45e7-8fcb-64038013c710 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:42.511716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387a66095d4c444ab9be10d6f42792a0', 'request_id': 'req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:42.520159 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387a66095d4c444ab9be10d6f42792a0', 'request_id': 'req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:42.521345 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1426/2834] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:42.528887 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387a66095d4c444ab9be10d6f42792a0', 'request_id': 'req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:42.544600 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19 admin admin] The token cache is being invalidate because project 387a66095d4c444ab9be10d6f42792a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:42:42.545093 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:42.546094 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-dd13f5f5-2b4c-4c18-ab63-a8dfd4801b19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:42.548113 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1409/2835] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:42:42 2022] DELETE /identity/v3/projects/387a66095d4c444ab9be10d6f42792a0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:42.553613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389a1f1-b136-4196-ae20-9e488ffcedc1 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:42.554146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389a1f1-b136-4196-ae20-9e488ffcedc1 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:42.554423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389a1f1-b136-4196-ae20-9e488ffcedc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:42.578142 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-7389a1f1-b136-4196-ae20-9e488ffcedc1 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:42.579543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-7389a1f1-b136-4196-ae20-9e488ffcedc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:42.648359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1427/2836] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:42.658078 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565db16-57e9-4eca-8236-a30d1ac68d1f None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:42.658564 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565db16-57e9-4eca-8236-a30d1ac68d1f None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:42.658666 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565db16-57e9-4eca-8236-a30d1ac68d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:42.684322 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6565db16-57e9-4eca-8236-a30d1ac68d1f None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:42.685583 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6565db16-57e9-4eca-8236-a30d1ac68d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:42.693875 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1410/2837] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:42.698401 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389a1f1-b136-4196-ae20-9e488ffcedc1 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:42.708041 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389a1f1-b136-4196-ae20-9e488ffcedc1 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:42.713669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7e1f8-f5f5-473f-a4be-9d8ae88210a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:42.714381 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7e1f8-f5f5-473f-a4be-9d8ae88210a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:42.714900 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7e1f8-f5f5-473f-a4be-9d8ae88210a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:42.715125 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7e1f8-f5f5-473f-a4be-9d8ae88210a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:42.716290 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7e1f8-f5f5-473f-a4be-9d8ae88210a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:42:42.716785 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7e1f8-f5f5-473f-a4be-9d8ae88210a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:42.718488 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7e1f8-f5f5-473f-a4be-9d8ae88210a0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:42.735939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1428/2838] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:42:42.737267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565db16-57e9-4eca-8236-a30d1ac68d1f None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:42.747043 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565db16-57e9-4eca-8236-a30d1ac68d1f None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:42.753734 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ff3cf-5a17-4136-9b3e-bcaefba8ab38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:42.754468 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ff3cf-5a17-4136-9b3e-bcaefba8ab38 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:42.754769 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ff3cf-5a17-4136-9b3e-bcaefba8ab38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:42.755040 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ff3cf-5a17-4136-9b3e-bcaefba8ab38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:42.755735 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ff3cf-5a17-4136-9b3e-bcaefba8ab38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:42.758049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ff3cf-5a17-4136-9b3e-bcaefba8ab38 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:42.791486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1411/2839] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:42:42 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:42:42.795168 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7e1f8-f5f5-473f-a4be-9d8ae88210a0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:42.807444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7e1f8-f5f5-473f-a4be-9d8ae88210a0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:42.858123 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=100 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:42.872607 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf26918-ac53-4094-b402-15996e578fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:42.873716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf26918-ac53-4094-b402-15996e578fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:42.874212 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf26918-ac53-4094-b402-15996e578fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:42.874743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf26918-ac53-4094-b402-15996e578fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:42.875933 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf26918-ac53-4094-b402-15996e578fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:42.879252 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf26918-ac53-4094-b402-15996e578fc0 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:42.915146 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1429/2840] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:42:42 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:42:42.916835 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ff3cf-5a17-4136-9b3e-bcaefba8ab38 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:42.928273 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ff3cf-5a17-4136-9b3e-bcaefba8ab38 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:42.934068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04cedb-0310-41b7-89e6-67c5283ac5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:42.934702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04cedb-0310-41b7-89e6-67c5283ac5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:42.934983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04cedb-0310-41b7-89e6-67c5283ac5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:42.935387 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04cedb-0310-41b7-89e6-67c5283ac5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:42.936049 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04cedb-0310-41b7-89e6-67c5283ac5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:42.937702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04cedb-0310-41b7-89e6-67c5283ac5d9 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:42.954631 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1412/2841] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:42:42.961823 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf26918-ac53-4094-b402-15996e578fc0 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:42.972507 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf26918-ac53-4094-b402-15996e578fc0 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:42.978920 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e70bd0-5c78-4dce-9d73-1888c47b2cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:42.979586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e70bd0-5c78-4dce-9d73-1888c47b2cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:42.979871 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e70bd0-5c78-4dce-9d73-1888c47b2cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:42.980152 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e70bd0-5c78-4dce-9d73-1888c47b2cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:42.980882 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e70bd0-5c78-4dce-9d73-1888c47b2cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:42.982651 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e70bd0-5c78-4dce-9d73-1888c47b2cb5 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:42.999053 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1430/2842] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:42:43.000517 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04cedb-0310-41b7-89e6-67c5283ac5d9 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:43.011604 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04cedb-0310-41b7-89e6-67c5283ac5d9 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:43.017744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8b381-378b-49bc-b1c2-288161cffb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:43.018419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8b381-378b-49bc-b1c2-288161cffb06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:43.018702 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8b381-378b-49bc-b1c2-288161cffb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:43.018981 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8b381-378b-49bc-b1c2-288161cffb06 admin admin] PATH_INFO: `/v3/projects/176f865ce7f542f4a7af595b17e2324a/users/057e6fd8802348b988baaa181ee163da/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:43.035648 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8b381-378b-49bc-b1c2-288161cffb06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=176f865ce7f542f4a7af595b17e2324a, user_id=057e6fd8802348b988baaa181ee163da, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:43.038395 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8b381-378b-49bc-b1c2-288161cffb06 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:43.053751 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1413/2843] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:42:42 2022] PUT /identity/v3/projects/176f865ce7f542f4a7af595b17e2324a/users/057e6fd8802348b988baaa181ee163da/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:42:43.059082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63304b55-702e-4098-bc28-58fb85590a35 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:43.059408 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63304b55-702e-4098-bc28-58fb85590a35 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:43.059646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63304b55-702e-4098-bc28-58fb85590a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:43.081360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-63304b55-702e-4098-bc28-58fb85590a35 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:43.082587 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-63304b55-702e-4098-bc28-58fb85590a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:43.144190 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1431/2844] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:43.154762 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8b381-378b-49bc-b1c2-288161cffb06 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:43.157376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-17f8b381-378b-49bc-b1c2-288161cffb06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:43.222935 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8b381-378b-49bc-b1c2-288161cffb06 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:43.227476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf31e60-6a0b-4e7c-b969-dec6d835c1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:43.228088 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf31e60-6a0b-4e7c-b969-dec6d835c1ee service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:43.228367 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf31e60-6a0b-4e7c-b969-dec6d835c1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:43.228637 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf31e60-6a0b-4e7c-b969-dec6d835c1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:43.238675 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf31e60-6a0b-4e7c-b969-dec6d835c1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:43.240183 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf31e60-6a0b-4e7c-b969-dec6d835c1ee service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:43.250108 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1414/2845] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:42:47.777169 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858b1b7-be25-4b7c-a62f-0e159facf721 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:47.777627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858b1b7-be25-4b7c-a62f-0e159facf721 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:47.777730 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858b1b7-be25-4b7c-a62f-0e159facf721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:47.814403 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-8858b1b7-be25-4b7c-a62f-0e159facf721 None None] MFA Rules not processed for user `057e6fd8802348b988baaa181ee163da`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:47.815721 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-8858b1b7-be25-4b7c-a62f-0e159facf721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:47.902371 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1432/2846] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:47.903903 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf31e60-6a0b-4e7c-b969-dec6d835c1ee service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:47.915658 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf31e60-6a0b-4e7c-b969-dec6d835c1ee service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:47.920721 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515f3c5-a967-4520-b5b2-8f285d3717e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:47.921542 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515f3c5-a967-4520-b5b2-8f285d3717e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:47.921818 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515f3c5-a967-4520-b5b2-8f285d3717e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:47.922086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515f3c5-a967-4520-b5b2-8f285d3717e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:47.922714 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515f3c5-a967-4520-b5b2-8f285d3717e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:47.924629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515f3c5-a967-4520-b5b2-8f285d3717e4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:47.955241 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1415/2847] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:42:47 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:42:47.959449 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858b1b7-be25-4b7c-a62f-0e159facf721 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:47.971347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858b1b7-be25-4b7c-a62f-0e159facf721 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:48.035391 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:48.043466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f9a61-016f-4de4-af96-f483a972c872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:48.044147 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f9a61-016f-4de4-af96-f483a972c872 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:48.044444 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f9a61-016f-4de4-af96-f483a972c872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:48.044914 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f9a61-016f-4de4-af96-f483a972c872 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:48.045852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f9a61-016f-4de4-af96-f483a972c872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:48.048084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f9a61-016f-4de4-af96-f483a972c872 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:48.073358 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1433/2848] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:42:47 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:42:48.077552 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515f3c5-a967-4520-b5b2-8f285d3717e4 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:48.090568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515f3c5-a967-4520-b5b2-8f285d3717e4 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:48.095750 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb726570-a669-44ae-9d7e-d4e2a2f75225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:48.096358 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb726570-a669-44ae-9d7e-d4e2a2f75225 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:48.096635 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb726570-a669-44ae-9d7e-d4e2a2f75225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:48.096954 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb726570-a669-44ae-9d7e-d4e2a2f75225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:48.097641 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb726570-a669-44ae-9d7e-d4e2a2f75225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:48.099265 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb726570-a669-44ae-9d7e-d4e2a2f75225 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:48.111415 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1416/2849] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:42:48.117743 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f9a61-016f-4de4-af96-f483a972c872 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:48.129397 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f9a61-016f-4de4-af96-f483a972c872 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:48.135085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c818a9-e38f-4bcb-b170-c439f0c03c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:48.135766 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c818a9-e38f-4bcb-b170-c439f0c03c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:48.136039 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c818a9-e38f-4bcb-b170-c439f0c03c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:48.136306 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c818a9-e38f-4bcb-b170-c439f0c03c40 admin admin] PATH_INFO: `/v3/projects/cce178327b0a4767b777f4b9102d4ff5/users/7142457ad40f4958b2449e380f738a7b/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:48.153105 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c818a9-e38f-4bcb-b170-c439f0c03c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cce178327b0a4767b777f4b9102d4ff5, user_id=7142457ad40f4958b2449e380f738a7b, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:48.156492 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c818a9-e38f-4bcb-b170-c439f0c03c40 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:48.171162 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1434/2850] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:42:48 2022] PUT /identity/v3/projects/cce178327b0a4767b777f4b9102d4ff5/users/7142457ad40f4958b2449e380f738a7b/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:48.175194 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb726570-a669-44ae-9d7e-d4e2a2f75225 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:48.185788 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb726570-a669-44ae-9d7e-d4e2a2f75225 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:48.206458 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:48.246148 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5586f-c5f0-451e-be73-91caa0f4bce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:48.246744 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5586f-c5f0-451e-be73-91caa0f4bce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:48.247015 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5586f-c5f0-451e-be73-91caa0f4bce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:48.247278 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5586f-c5f0-451e-be73-91caa0f4bce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:48.247894 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5586f-c5f0-451e-be73-91caa0f4bce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:48.249563 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5586f-c5f0-451e-be73-91caa0f4bce4 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:48.259800 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1417/2851] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:42:48.264056 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c818a9-e38f-4bcb-b170-c439f0c03c40 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:48.274858 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c818a9-e38f-4bcb-b170-c439f0c03c40 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:48.279727 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035abc1-e790-49a0-bb26-ae0fb6d7f654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:48.280376 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035abc1-e790-49a0-bb26-ae0fb6d7f654 admin admin] REQUEST_METHOD: `PUT` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:48.280669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035abc1-e790-49a0-bb26-ae0fb6d7f654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:48.281295 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035abc1-e790-49a0-bb26-ae0fb6d7f654 admin admin] PATH_INFO: `/v3/projects/cce178327b0a4767b777f4b9102d4ff5/users/7142457ad40f4958b2449e380f738a7b/roles/c541c0c93cc7434ab3420744ede203a9` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:48.283264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035abc1-e790-49a0-bb26-ae0fb6d7f654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cce178327b0a4767b777f4b9102d4ff5, user_id=7142457ad40f4958b2449e380f738a7b, role_id=c541c0c93cc7434ab3420744ede203a9)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:48.285959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035abc1-e790-49a0-bb26-ae0fb6d7f654 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:48.296498 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1435/2852] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:42:48 2022] PUT /identity/v3/projects/cce178327b0a4767b777f4b9102d4ff5/users/7142457ad40f4958b2449e380f738a7b/roles/c541c0c93cc7434ab3420744ede203a9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:42:53.216316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3c07e-504e-432d-95eb-523bd1c449ad None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:53.217297 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3c07e-504e-432d-95eb-523bd1c449ad None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:53.217765 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3c07e-504e-432d-95eb-523bd1c449ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:53.255621 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-7fd3c07e-504e-432d-95eb-523bd1c449ad None None] MFA Rules not processed for user `7142457ad40f4958b2449e380f738a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:53.259082 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-7fd3c07e-504e-432d-95eb-523bd1c449ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:53.368692 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1418/2853] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:53.385093 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035abc1-e790-49a0-bb26-ae0fb6d7f654 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:53.387021 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a035abc1-e790-49a0-bb26-ae0fb6d7f654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:53.479758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035abc1-e790-49a0-bb26-ae0fb6d7f654 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:53.485672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febad73a-94b2-4c28-8a33-469d702731a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:53.486392 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febad73a-94b2-4c28-8a33-469d702731a8 service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:53.486733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febad73a-94b2-4c28-8a33-469d702731a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:53.487068 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febad73a-94b2-4c28-8a33-469d702731a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:53.576690 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febad73a-94b2-4c28-8a33-469d702731a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:53.580012 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febad73a-94b2-4c28-8a33-469d702731a8 service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:53.597947 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1436/2854] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:54.708541 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d209e02-96e7-4d6a-b66f-55d1f97cb481 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:54.709071 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d209e02-96e7-4d6a-b66f-55d1f97cb481 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:54.709510 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d209e02-96e7-4d6a-b66f-55d1f97cb481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:54.730495 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-6d209e02-96e7-4d6a-b66f-55d1f97cb481 None None] MFA Rules not processed for user `057e6fd8802348b988baaa181ee163da`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:54.731711 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6d209e02-96e7-4d6a-b66f-55d1f97cb481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:54.738722 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1419/2855] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:42:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:58.093389 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c397b7-c3b6-46f2-a759-1ec16b2f7d38 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:58.094019 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c397b7-c3b6-46f2-a759-1ec16b2f7d38 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:58.094340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c397b7-c3b6-46f2-a759-1ec16b2f7d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:58.116374 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-a9c397b7-c3b6-46f2-a759-1ec16b2f7d38 None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:58.117855 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a9c397b7-c3b6-46f2-a759-1ec16b2f7d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:58.163968 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:58.183089 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1437/2856] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:42:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:58.194960 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d209e02-96e7-4d6a-b66f-55d1f97cb481 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:58.196485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-6d209e02-96e7-4d6a-b66f-55d1f97cb481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:58.276209 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d209e02-96e7-4d6a-b66f-55d1f97cb481 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:58.282832 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b7b43-f271-40bb-9878-e47e178002ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:58.283446 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b7b43-f271-40bb-9878-e47e178002ff service swift] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:58.283719 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b7b43-f271-40bb-9878-e47e178002ff service swift] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:58.283981 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b7b43-f271-40bb-9878-e47e178002ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:58.294025 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b7b43-f271-40bb-9878-e47e178002ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:58.295690 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b7b43-f271-40bb-9878-e47e178002ff service swift] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:58.305001 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1420/2857] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:42:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:42:58.893802 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649d479-ab02-41b5-8363-642504633ef6 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:58.894130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649d479-ab02-41b5-8363-642504633ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:58.894370 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649d479-ab02-41b5-8363-642504633ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:58.916044 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3649d479-ab02-41b5-8363-642504633ef6 None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:58.917313 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3649d479-ab02-41b5-8363-642504633ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:58.925482 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1438/2858] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 24 12:42:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:58.937868 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b7b43-f271-40bb-9878-e47e178002ff service swift] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:58.948881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b7b43-f271-40bb-9878-e47e178002ff service swift] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:58.955392 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17831bfc-b050-41ec-94dc-1fc8b6c7b5b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:58.956268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17831bfc-b050-41ec-94dc-1fc8b6c7b5b5 service swift] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:58.956525 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17831bfc-b050-41ec-94dc-1fc8b6c7b5b5 service swift] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:58.956992 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17831bfc-b050-41ec-94dc-1fc8b6c7b5b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:58.969937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17831bfc-b050-41ec-94dc-1fc8b6c7b5b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:58.971656 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17831bfc-b050-41ec-94dc-1fc8b6c7b5b5 service swift] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:58.984324 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1421/2859] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:42:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:42:59.237061 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b96dbb-7500-40d9-8fcc-295dedb8d99b None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:59.237575 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b96dbb-7500-40d9-8fcc-295dedb8d99b None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:59.237679 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b96dbb-7500-40d9-8fcc-295dedb8d99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:59.258637 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d2b96dbb-7500-40d9-8fcc-295dedb8d99b None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:59.259887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d2b96dbb-7500-40d9-8fcc-295dedb8d99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:59.267983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1439/2860] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 12:42:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:59.280733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17831bfc-b050-41ec-94dc-1fc8b6c7b5b5 service swift] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:59.291531 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17831bfc-b050-41ec-94dc-1fc8b6c7b5b5 service swift] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:59.296972 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbac5e-d7e6-4470-8211-371e5183709b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:59.297553 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbac5e-d7e6-4470-8211-371e5183709b service swift] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:59.297827 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbac5e-d7e6-4470-8211-371e5183709b service swift] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:59.298091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbac5e-d7e6-4470-8211-371e5183709b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:59.307558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbac5e-d7e6-4470-8211-371e5183709b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:59.309929 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbac5e-d7e6-4470-8211-371e5183709b service swift] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:59.315444 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:59.320389 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1422/2861] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:42:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:42:59.672446 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03b8a0-77a9-4f36-954a-c9a7154601ac None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:59.672774 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03b8a0-77a9-4f36-954a-c9a7154601ac None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:59.673071 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03b8a0-77a9-4f36-954a-c9a7154601ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:59.693564 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-bc03b8a0-77a9-4f36-954a-c9a7154601ac None None] MFA Rules not processed for user `34a3cadee8ea4b3e9dff0bc23dfe9dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:42:59.694768 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bc03b8a0-77a9-4f36-954a-c9a7154601ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:59.702606 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1440/2862] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 24 12:42:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:42:59.715688 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbac5e-d7e6-4470-8211-371e5183709b service swift] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:59.726720 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbac5e-d7e6-4470-8211-371e5183709b service swift] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:59.732115 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b246fa-b2fe-47a2-b43c-c2302e64a7dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfec040b09c9415b9e89cd28c67aa5df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:59.732772 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b246fa-b2fe-47a2-b43c-c2302e64a7dd service swift] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:59.733050 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b246fa-b2fe-47a2-b43c-c2302e64a7dd service swift] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:59.733318 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b246fa-b2fe-47a2-b43c-c2302e64a7dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:59.743130 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b246fa-b2fe-47a2-b43c-c2302e64a7dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:59.744889 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b246fa-b2fe-47a2-b43c-c2302e64a7dd service swift] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:59.753864 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1423/2863] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 12:42:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:43:02.158216 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03b8a0-77a9-4f36-954a-c9a7154601ac None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:02.159781 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bc03b8a0-77a9-4f36-954a-c9a7154601ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:02.233107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03b8a0-77a9-4f36-954a-c9a7154601ac None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:02.238088 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:02.238627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:02.238905 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:02.239178 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] PATH_INFO: `/v3/users/7159c045925f48e8a03594a7d38e9d27` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:02.240340 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7159c045925f48e8a03594a7d38e9d27)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:02.242864 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:02.301829 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7159c045925f48e8a03594a7d38e9d27'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:02.302506 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7159c045925f48e8a03594a7d38e9d27'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:02.303128 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7159c045925f48e8a03594a7d38e9d27'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:02.315959 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7159c045925f48e8a03594a7d38e9d27'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:02.327640 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7159c045925f48e8a03594a7d38e9d27'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:02.341522 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1441/2864] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:43:02 2022] DELETE /identity/v3/users/7159c045925f48e8a03594a7d38e9d27 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:43:02.346203 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02945fed-6023-418c-97f1-a640a076ae15 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:02.346730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02945fed-6023-418c-97f1-a640a076ae15 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:02.346730 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02945fed-6023-418c-97f1-a640a076ae15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:02.365661 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-02945fed-6023-418c-97f1-a640a076ae15 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:43:02.367062 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-02945fed-6023-418c-97f1-a640a076ae15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:02.437127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1424/2865] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:43:02.449347 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:02.451733 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:02.522310 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d0c38-a417-41b1-86ca-7c5a3b04f10c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:02.526228 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27432b-11f5-43aa-b7b8-94ad3ff849c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:02.526857 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27432b-11f5-43aa-b7b8-94ad3ff849c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:02.527130 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27432b-11f5-43aa-b7b8-94ad3ff849c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:02.527395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27432b-11f5-43aa-b7b8-94ad3ff849c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:02.533794 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:02.537891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27432b-11f5-43aa-b7b8-94ad3ff849c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:02.539732 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27432b-11f5-43aa-b7b8-94ad3ff849c9 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:02.550270 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1442/2866] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:02.742749 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02945fed-6023-418c-97f1-a640a076ae15 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:02.744519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-02945fed-6023-418c-97f1-a640a076ae15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:02.757272 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02945fed-6023-418c-97f1-a640a076ae15 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:02.763445 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff9f30-967f-48ff-9d5c-e90a27346317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:02.764091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff9f30-967f-48ff-9d5c-e90a27346317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:02.764360 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff9f30-967f-48ff-9d5c-e90a27346317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:02.764622 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff9f30-967f-48ff-9d5c-e90a27346317 admin admin] PATH_INFO: `/v3/projects/c5da2f4e102b49438b2b80ba75d1bf0c` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:02.766014 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff9f30-967f-48ff-9d5c-e90a27346317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5da2f4e102b49438b2b80ba75d1bf0c)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:02.768073 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff9f30-967f-48ff-9d5c-e90a27346317 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:02.822733 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-10ff9f30-967f-48ff-9d5c-e90a27346317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5da2f4e102b49438b2b80ba75d1bf0c', 'request_id': 'req-10ff9f30-967f-48ff-9d5c-e90a27346317'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:02.834485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-10ff9f30-967f-48ff-9d5c-e90a27346317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5da2f4e102b49438b2b80ba75d1bf0c', 'request_id': 'req-10ff9f30-967f-48ff-9d5c-e90a27346317'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:02.842248 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-10ff9f30-967f-48ff-9d5c-e90a27346317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5da2f4e102b49438b2b80ba75d1bf0c', 'request_id': 'req-10ff9f30-967f-48ff-9d5c-e90a27346317'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:02.853916 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-10ff9f30-967f-48ff-9d5c-e90a27346317 admin admin] The token cache is being invalidate because project c5da2f4e102b49438b2b80ba75d1bf0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:43:02.854206 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-10ff9f30-967f-48ff-9d5c-e90a27346317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:02.854904 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-10ff9f30-967f-48ff-9d5c-e90a27346317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:02.856449 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1425/2867] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:43:02 2022] DELETE /identity/v3/projects/c5da2f4e102b49438b2b80ba75d1bf0c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:43:02.861742 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783b5a3-76d5-4831-a169-78ce79c10659 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:02.862023 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783b5a3-76d5-4831-a169-78ce79c10659 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:02.862289 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783b5a3-76d5-4831-a169-78ce79c10659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:02.883217 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-d783b5a3-76d5-4831-a169-78ce79c10659 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:43:02.884418 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-d783b5a3-76d5-4831-a169-78ce79c10659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:02.950949 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1443/2868] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:43:02.960021 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48128ffa-f6e2-4ce6-92f6-740245f92469 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:02.960289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48128ffa-f6e2-4ce6-92f6-740245f92469 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:02.960601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48128ffa-f6e2-4ce6-92f6-740245f92469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:02.986673 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-48128ffa-f6e2-4ce6-92f6-740245f92469 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:43:02.988578 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-48128ffa-f6e2-4ce6-92f6-740245f92469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:02.999559 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1426/2869] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:43:03.004154 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783b5a3-76d5-4831-a169-78ce79c10659 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:03.014716 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783b5a3-76d5-4831-a169-78ce79c10659 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:03.019725 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fa759-980c-4449-9291-2dc572bdc27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:03.020336 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa759-980c-4449-9291-2dc572bdc27a admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:03.020613 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa759-980c-4449-9291-2dc572bdc27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:03.020939 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa759-980c-4449-9291-2dc572bdc27a admin admin] PATH_INFO: `/v3/domains` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:03.021586 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fa759-980c-4449-9291-2dc572bdc27a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:43:03.021890 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fa759-980c-4449-9291-2dc572bdc27a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:03.023736 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fa759-980c-4449-9291-2dc572bdc27a admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:03.036134 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1444/2870] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 12:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:43:03.041053 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48128ffa-f6e2-4ce6-92f6-740245f92469 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:03.051411 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48128ffa-f6e2-4ce6-92f6-740245f92469 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:03.056696 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3147dd0-d5a5-445b-b32c-5916a6012a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:03.057316 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3147dd0-d5a5-445b-b32c-5916a6012a28 admin admin] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:03.057592 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3147dd0-d5a5-445b-b32c-5916a6012a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:03.057860 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3147dd0-d5a5-445b-b32c-5916a6012a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:03.058486 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3147dd0-d5a5-445b-b32c-5916a6012a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:03.060420 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3147dd0-d5a5-445b-b32c-5916a6012a28 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:03.089536 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1427/2871] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 12:43:03 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:43:03.093983 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fa759-980c-4449-9291-2dc572bdc27a admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:03.105473 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fa759-980c-4449-9291-2dc572bdc27a admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:03.166987 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80adf657-ccca-44da-8fc9-f517cd01a612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:03.167830 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80adf657-ccca-44da-8fc9-f517cd01a612 admin admin] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:03.168109 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80adf657-ccca-44da-8fc9-f517cd01a612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:03.168383 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80adf657-ccca-44da-8fc9-f517cd01a612 admin admin] PATH_INFO: `/v3/users` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:03.169107 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80adf657-ccca-44da-8fc9-f517cd01a612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:03.171815 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80adf657-ccca-44da-8fc9-f517cd01a612 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:03.203333 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1445/2872] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 12:43:03 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:43:03.210369 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3147dd0-d5a5-445b-b32c-5916a6012a28 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:03.221782 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3147dd0-d5a5-445b-b32c-5916a6012a28 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:03.227728 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69c94-bfe6-465e-acce-55ca8541e165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:03.228376 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69c94-bfe6-465e-acce-55ca8541e165 admin admin] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:03.228629 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69c94-bfe6-465e-acce-55ca8541e165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:03.228939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69c94-bfe6-465e-acce-55ca8541e165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:03.229587 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c69c94-bfe6-465e-acce-55ca8541e165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:03.231210 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c69c94-bfe6-465e-acce-55ca8541e165 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:03.241997 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1428/2873] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:43:03.246805 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80adf657-ccca-44da-8fc9-f517cd01a612 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:03.261082 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80adf657-ccca-44da-8fc9-f517cd01a612 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:03.268973 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76268a5-fe74-47c2-81a0-c58f244783d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:03.269824 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76268a5-fe74-47c2-81a0-c58f244783d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:03.270161 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76268a5-fe74-47c2-81a0-c58f244783d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:03.270504 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76268a5-fe74-47c2-81a0-c58f244783d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:03.271528 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76268a5-fe74-47c2-81a0-c58f244783d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:03.273646 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76268a5-fe74-47c2-81a0-c58f244783d1 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:03.285849 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1446/2874] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 12:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:43:03.291307 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69c94-bfe6-465e-acce-55ca8541e165 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:03.302130 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69c94-bfe6-465e-acce-55ca8541e165 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:03.307630 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fb355-f279-4eb4-a7db-73b004d34aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:03.308289 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb355-f279-4eb4-a7db-73b004d34aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:03.308568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb355-f279-4eb4-a7db-73b004d34aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:03.308946 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb355-f279-4eb4-a7db-73b004d34aaa admin admin] PATH_INFO: `/v3/projects/b5463ee93df54d698524bc67df7978fd/users/6b6c6f7561074fb8a78d8ef54d98bc31/roles/3647def0715f4fd4991f82b06815ee38` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:03.324445 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fb355-f279-4eb4-a7db-73b004d34aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5463ee93df54d698524bc67df7978fd, user_id=6b6c6f7561074fb8a78d8ef54d98bc31, role_id=3647def0715f4fd4991f82b06815ee38)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:03.327147 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fb355-f279-4eb4-a7db-73b004d34aaa admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:03.343337 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1429/2875] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 12:43:03 2022] PUT /identity/v3/projects/b5463ee93df54d698524bc67df7978fd/users/6b6c6f7561074fb8a78d8ef54d98bc31/roles/3647def0715f4fd4991f82b06815ee38 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:43:03.354032 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c9722-6ee9-4011-b611-755a68267c38 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:03.354317 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c9722-6ee9-4011-b611-755a68267c38 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:03.354590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c9722-6ee9-4011-b611-755a68267c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:03.384978 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-3a7c9722-6ee9-4011-b611-755a68267c38 None None] MFA Rules not processed for user `6b6c6f7561074fb8a78d8ef54d98bc31`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:43:03.386229 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3a7c9722-6ee9-4011-b611-755a68267c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:03.438188 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:03.457852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1447/2876] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:43:03 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:43:03.470163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fb355-f279-4eb4-a7db-73b004d34aaa admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:03.472162 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-487fb355-f279-4eb4-a7db-73b004d34aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:03.483644 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:03.546080 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fb355-f279-4eb4-a7db-73b004d34aaa admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:03.550917 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a43e1-efe2-49db-9df2-bc78fa4e9c70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:03.551716 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a43e1-efe2-49db-9df2-bc78fa4e9c70 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:03.552024 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a43e1-efe2-49db-9df2-bc78fa4e9c70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:03.552145 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a43e1-efe2-49db-9df2-bc78fa4e9c70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:03.562409 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a43e1-efe2-49db-9df2-bc78fa4e9c70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:03.563879 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a43e1-efe2-49db-9df2-bc78fa4e9c70 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:03.573572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1430/2877] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:03 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:43:11.808096 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c9722-6ee9-4011-b611-755a68267c38 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:11.876000 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c9722-6ee9-4011-b611-755a68267c38 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:11.882202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:11.882841 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:11.883122 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:11.883438 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] PATH_INFO: `/v3/users/6b6c6f7561074fb8a78d8ef54d98bc31` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:11.884695 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b6c6f7561074fb8a78d8ef54d98bc31)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:11.886664 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:11.950559 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b6c6f7561074fb8a78d8ef54d98bc31'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:11.951206 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b6c6f7561074fb8a78d8ef54d98bc31'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:11.951811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b6c6f7561074fb8a78d8ef54d98bc31'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:11.964330 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b6c6f7561074fb8a78d8ef54d98bc31'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:11.976881 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b6c6f7561074fb8a78d8ef54d98bc31'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:11.990694 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1448/2878] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:43:11 2022] DELETE /identity/v3/users/6b6c6f7561074fb8a78d8ef54d98bc31 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:43:11.995659 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b51ea7-d1c3-4946-bc49-b78c6ccddab2 None None] REQUEST_METHOD: `POST` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:11.996163 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b51ea7-d1c3-4946-bc49-b78c6ccddab2 None None] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:11.996262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b51ea7-d1c3-4946-bc49-b78c6ccddab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:12.015652 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.auth.core [None req-37b51ea7-d1c3-4946-bc49-b78c6ccddab2 None None] MFA Rules not processed for user `fe15d570e61b4d028bf4767532315b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:43:12.016881 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-37b51ea7-d1c3-4946-bc49-b78c6ccddab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:12.079618 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1431/2879] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:43:12.092238 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:12.093932 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:12.154611 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff33cf-d643-4c91-8919-5db2c21ff0e7 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:12.158627 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a67b5a-112f-4b40-a41e-6afa04a17291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:12.159236 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a67b5a-112f-4b40-a41e-6afa04a17291 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:12.159519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a67b5a-112f-4b40-a41e-6afa04a17291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:12.159789 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a67b5a-112f-4b40-a41e-6afa04a17291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:12.169767 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a67b5a-112f-4b40-a41e-6afa04a17291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:12.171207 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a67b5a-112f-4b40-a41e-6afa04a17291 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:12.181691 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1449/2880] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:12.466680 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b51ea7-d1c3-4946-bc49-b78c6ccddab2 None None] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:12.468185 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-37b51ea7-d1c3-4946-bc49-b78c6ccddab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:12.482136 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b51ea7-d1c3-4946-bc49-b78c6ccddab2 None None] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:12.489105 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:12.489993 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:12.490594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:12.490869 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] PATH_INFO: `/v3/projects/b5463ee93df54d698524bc67df7978fd` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:12.492939 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5463ee93df54d698524bc67df7978fd)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:12.495179 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:12.551586 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5463ee93df54d698524bc67df7978fd', 'request_id': 'req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:12.562718 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5463ee93df54d698524bc67df7978fd', 'request_id': 'req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:12.569598 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5463ee93df54d698524bc67df7978fd', 'request_id': 'req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:12.581350 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] The token cache is being invalidate because project b5463ee93df54d698524bc67df7978fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:43:12.581579 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:12.582303 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:12.584010 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1432/2881] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:43:12 2022] DELETE /identity/v3/projects/b5463ee93df54d698524bc67df7978fd => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:43:17.186998 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a67b5a-112f-4b40-a41e-6afa04a17291 service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:17.188502 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a2a67b5a-112f-4b40-a41e-6afa04a17291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:17.201688 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a67b5a-112f-4b40-a41e-6afa04a17291 service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:17.212776 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:17.265972 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377245b2-9538-45ab-938c-299a19c5fadc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:17.267368 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377245b2-9538-45ab-938c-299a19c5fadc service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:17.267844 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377245b2-9538-45ab-938c-299a19c5fadc service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:17.268318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377245b2-9538-45ab-938c-299a19c5fadc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:17.271318 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-377245b2-9538-45ab-938c-299a19c5fadc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:17.282322 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377245b2-9538-45ab-938c-299a19c5fadc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:17.283839 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377245b2-9538-45ab-938c-299a19c5fadc service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:17.292503 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1450/2882] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:22.578346 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:27.288911 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=21 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:28.160636 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:28.162193 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:28.224824 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee195fc-d3a7-44f8-9e74-3f6017285ae3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:28.229086 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efe53a-3ef4-4341-b03d-55a365cff3c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b53c9f4ff842e08ec41f53147d0431', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:28.229683 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efe53a-3ef4-4341-b03d-55a365cff3c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:28.229953 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efe53a-3ef4-4341-b03d-55a365cff3c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:28.230216 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efe53a-3ef4-4341-b03d-55a365cff3c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:28.232364 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-97efe53a-3ef4-4341-b03d-55a365cff3c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:28.294323 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efe53a-3ef4-4341-b03d-55a365cff3c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:28.295803 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efe53a-3ef4-4341-b03d-55a365cff3c2 service cinder] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:28.305501 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1433/2883] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:34.517684 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377245b2-9538-45ab-938c-299a19c5fadc service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:34.519323 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-377245b2-9538-45ab-938c-299a19c5fadc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:34.531807 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377245b2-9538-45ab-938c-299a19c5fadc service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:34.537084 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7def2-db4d-42d6-a242-3e4e3851d525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:34.537623 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7def2-db4d-42d6-a242-3e4e3851d525 service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:34.537894 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7def2-db4d-42d6-a242-3e4e3851d525 service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:34.538166 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7def2-db4d-42d6-a242-3e4e3851d525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:34.540369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-54b7def2-db4d-42d6-a242-3e4e3851d525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:34.609066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7def2-db4d-42d6-a242-3e4e3851d525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:34.610519 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7def2-db4d-42d6-a242-3e4e3851d525 service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:34.620202 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1451/2884] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:36.261670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efe53a-3ef4-4341-b03d-55a365cff3c2 service cinder] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:36.263268 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-97efe53a-3ef4-4341-b03d-55a365cff3c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:36.281367 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290c8a1-7c7c-49b5-94ce-f0f85de0ca47 None None] REQUEST_METHOD: `POST` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:36.282274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290c8a1-7c7c-49b5-94ce-f0f85de0ca47 None None] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:36.282274 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290c8a1-7c7c-49b5-94ce-f0f85de0ca47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:36.304303 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.auth.core [None req-f290c8a1-7c7c-49b5-94ce-f0f85de0ca47 None None] MFA Rules not processed for user `ae1cd97c879141e2b4c1527b5964d054`. Rule list: `[]` (Enabled: `True`). {{(pid=81635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:43:36.306500 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f290c8a1-7c7c-49b5-94ce-f0f85de0ca47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:36.316368 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1452/2885] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 12:43:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:43:36.347505 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efe53a-3ef4-4341-b03d-55a365cff3c2 service cinder] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:36.353455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd7855-0d2e-4353-8b5e-7148da0116dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:36.354263 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd7855-0d2e-4353-8b5e-7148da0116dd service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:36.354964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd7855-0d2e-4353-8b5e-7148da0116dd service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:36.354964 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd7855-0d2e-4353-8b5e-7148da0116dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:36.357091 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5cbd7855-0d2e-4353-8b5e-7148da0116dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:36.372932 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd7855-0d2e-4353-8b5e-7148da0116dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:36.374967 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd7855-0d2e-4353-8b5e-7148da0116dd service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:36.386820 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1434/2886] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:36.803069 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290c8a1-7c7c-49b5-94ce-f0f85de0ca47 None None] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:36.804552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-f290c8a1-7c7c-49b5-94ce-f0f85de0ca47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:36.817672 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290c8a1-7c7c-49b5-94ce-f0f85de0ca47 None None] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:36.822240 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cf040-a680-4c74-b44d-81d62452c9cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:36.822867 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cf040-a680-4c74-b44d-81d62452c9cf service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:36.827885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cf040-a680-4c74-b44d-81d62452c9cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:36.827885 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cf040-a680-4c74-b44d-81d62452c9cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:36.830784 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-462cf040-a680-4c74-b44d-81d62452c9cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:36.845237 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cf040-a680-4c74-b44d-81d62452c9cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:36.846765 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cf040-a680-4c74-b44d-81d62452c9cf service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:36.855705 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1453/2887] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:37.785693 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd7855-0d2e-4353-8b5e-7148da0116dd service nova] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:37.787199 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5cbd7855-0d2e-4353-8b5e-7148da0116dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:37.832753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=100 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:37.858670 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd7855-0d2e-4353-8b5e-7148da0116dd service nova] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:37.864942 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027ed0d-6402-4a9b-8cda-ac29474c4bae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:37.865576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027ed0d-6402-4a9b-8cda-ac29474c4bae service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:37.865854 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027ed0d-6402-4a9b-8cda-ac29474c4bae service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:37.866123 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027ed0d-6402-4a9b-8cda-ac29474c4bae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:37.868135 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-c027ed0d-6402-4a9b-8cda-ac29474c4bae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:37.880892 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027ed0d-6402-4a9b-8cda-ac29474c4bae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:37.882613 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027ed0d-6402-4a9b-8cda-ac29474c4bae service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:37.893560 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1435/2888] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:40.458277 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cf040-a680-4c74-b44d-81d62452c9cf service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:40.460496 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-462cf040-a680-4c74-b44d-81d62452c9cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:40.479717 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cf040-a680-4c74-b44d-81d62452c9cf service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:40.486685 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbba12-fdb0-4098-b8d2-8bb9bf11b7ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:40.487292 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbba12-fdb0-4098-b8d2-8bb9bf11b7ae service nova] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:40.487574 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbba12-fdb0-4098-b8d2-8bb9bf11b7ae service nova] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:40.487848 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbba12-fdb0-4098-b8d2-8bb9bf11b7ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:40.489982 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-99dbba12-fdb0-4098-b8d2-8bb9bf11b7ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:40.507073 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbba12-fdb0-4098-b8d2-8bb9bf11b7ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:40.508891 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbba12-fdb0-4098-b8d2-8bb9bf11b7ae service nova] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:40.521235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1454/2889] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:47.888576 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=14 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:50.081438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027ed0d-6402-4a9b-8cda-ac29474c4bae service placement] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:50.095784 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027ed0d-6402-4a9b-8cda-ac29474c4bae service placement] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:50.102328 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeff5f6-394c-4c18-82b2-70ca9528f4b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e65026e01df242e7be11ed24214a4efa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:50.102966 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeff5f6-394c-4c18-82b2-70ca9528f4b6 service nova] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:50.103239 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeff5f6-394c-4c18-82b2-70ca9528f4b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:50.103503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeff5f6-394c-4c18-82b2-70ca9528f4b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:50.105485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-ebeff5f6-394c-4c18-82b2-70ca9528f4b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:50.118834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeff5f6-394c-4c18-82b2-70ca9528f4b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:50.120491 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeff5f6-394c-4c18-82b2-70ca9528f4b6 service nova] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:50.132125 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1436/2890] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:50.515862 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=45 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:44:00.127167 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=6 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:45:48.973391 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbba12-fdb0-4098-b8d2-8bb9bf11b7ae service nova] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:48.984991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbba12-fdb0-4098-b8d2-8bb9bf11b7ae service nova] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:48.989080 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be104b-193b-4b85-b319-928badf748bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:48.989689 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be104b-193b-4b85-b319-928badf748bf service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:48.989961 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be104b-193b-4b85-b319-928badf748bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:48.990226 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be104b-193b-4b85-b319-928badf748bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:48.992096 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-e5be104b-193b-4b85-b319-928badf748bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:49.003999 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be104b-193b-4b85-b319-928badf748bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:49.005567 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be104b-193b-4b85-b319-928badf748bf service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:49.015704 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1455/2891] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:45:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:45:49.151498 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeff5f6-394c-4c18-82b2-70ca9528f4b6 service nova] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:49.162538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeff5f6-394c-4c18-82b2-70ca9528f4b6 service nova] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:49.166897 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40bcb2-cd2d-4f11-a924-eb2d056bce29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:49.167907 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40bcb2-cd2d-4f11-a924-eb2d056bce29 service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:49.168504 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40bcb2-cd2d-4f11-a924-eb2d056bce29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:49.169069 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40bcb2-cd2d-4f11-a924-eb2d056bce29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:49.171729 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-0b40bcb2-cd2d-4f11-a924-eb2d056bce29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:49.184020 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40bcb2-cd2d-4f11-a924-eb2d056bce29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:49.185558 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40bcb2-cd2d-4f11-a924-eb2d056bce29 service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:49.195588 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1437/2892] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:45:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:45:49.274705 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be104b-193b-4b85-b319-928badf748bf service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:49.284524 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be104b-193b-4b85-b319-928badf748bf service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:49.288860 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28ddd9-db9d-4861-8463-25eda1cd84af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:49.289466 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28ddd9-db9d-4861-8463-25eda1cd84af service neutron] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:49.289761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28ddd9-db9d-4861-8463-25eda1cd84af service neutron] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:49.290030 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28ddd9-db9d-4861-8463-25eda1cd84af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:49.291745 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3f28ddd9-db9d-4861-8463-25eda1cd84af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:49.306706 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28ddd9-db9d-4861-8463-25eda1cd84af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:49.308140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28ddd9-db9d-4861-8463-25eda1cd84af service neutron] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:49.320369 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1456/2893] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:45:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:45:51.514862 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40bcb2-cd2d-4f11-a924-eb2d056bce29 service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:51.525354 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40bcb2-cd2d-4f11-a924-eb2d056bce29 service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:51.530207 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1ee1a-f0eb-4667-ace6-0a768d02fbbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e18461c8cc4497eae7a83d9ad1448f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:51.530948 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1ee1a-f0eb-4667-ace6-0a768d02fbbb service neutron] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:51.531518 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1ee1a-f0eb-4667-ace6-0a768d02fbbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:51.531886 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1ee1a-f0eb-4667-ace6-0a768d02fbbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:51.533895 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d2f1ee1a-f0eb-4667-ace6-0a768d02fbbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:51.604888 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1ee1a-f0eb-4667-ace6-0a768d02fbbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:51.606388 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1ee1a-f0eb-4667-ace6-0a768d02fbbb service neutron] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:51.616995 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1438/2894] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:45:54.875163 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28ddd9-db9d-4861-8463-25eda1cd84af service neutron] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:54.876943 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-3f28ddd9-db9d-4861-8463-25eda1cd84af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:54.894410 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28ddd9-db9d-4861-8463-25eda1cd84af service neutron] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:54.901235 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8061de7-4638-4697-aa1d-c87ce2eb3dde service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:54.901856 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8061de7-4638-4697-aa1d-c87ce2eb3dde service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:54.902136 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8061de7-4638-4697-aa1d-c87ce2eb3dde service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:54.902409 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8061de7-4638-4697-aa1d-c87ce2eb3dde service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:54.904614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-a8061de7-4638-4697-aa1d-c87ce2eb3dde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:54.918924 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8061de7-4638-4697-aa1d-c87ce2eb3dde service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:54.921453 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8061de7-4638-4697-aa1d-c87ce2eb3dde service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:54.941423 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1457/2895] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:45:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:45:58.511441 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1ee1a-f0eb-4667-ace6-0a768d02fbbb service neutron] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:58.513066 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-d2f1ee1a-f0eb-4667-ace6-0a768d02fbbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:58.526204 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1ee1a-f0eb-4667-ace6-0a768d02fbbb service neutron] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:58.531408 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:58.532262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:58.532544 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:58.533171 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] PATH_INFO: `/v3/users/ae1cd97c879141e2b4c1527b5964d054` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:58.534572 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae1cd97c879141e2b4c1527b5964d054)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:58.536841 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:58.589587 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae1cd97c879141e2b4c1527b5964d054'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.590251 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae1cd97c879141e2b4c1527b5964d054'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.590907 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae1cd97c879141e2b4c1527b5964d054'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.603928 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae1cd97c879141e2b4c1527b5964d054'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.613041 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae1cd97c879141e2b4c1527b5964d054'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.626068 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1439/2896] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:45:58 2022] DELETE /identity/v3/users/ae1cd97c879141e2b4c1527b5964d054 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:45:58.815703 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8061de7-4638-4697-aa1d-c87ce2eb3dde service placement] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:58.829714 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8061de7-4638-4697-aa1d-c87ce2eb3dde service placement] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:58.894797 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:58.895436 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:58.895715 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:58.895985 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] PATH_INFO: `/v3/projects/e112083132a9492089fafdae9e34f4ff` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:58.897143 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e112083132a9492089fafdae9e34f4ff)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:58.899150 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:58.949241 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e112083132a9492089fafdae9e34f4ff', 'request_id': 'req-10111a47-87c9-4804-aa26-9fc9857e865c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.958695 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e112083132a9492089fafdae9e34f4ff', 'request_id': 'req-10111a47-87c9-4804-aa26-9fc9857e865c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.965115 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e112083132a9492089fafdae9e34f4ff', 'request_id': 'req-10111a47-87c9-4804-aa26-9fc9857e865c'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.976285 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] The token cache is being invalidate because project e112083132a9492089fafdae9e34f4ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:45:58.976562 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.977264 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.979090 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1458/2897] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:45:58 2022] DELETE /identity/v3/projects/e112083132a9492089fafdae9e34f4ff => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:45:58.983374 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:58.984937 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:59.000077 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71df97-df9a-4d8f-a1c5-967fb9a287d3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:59.051485 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:45:59.062568 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:59.063443 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:59.063753 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:59.064044 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] PATH_INFO: `/v3/users/852082d3b62f49b48881b81bf3ce1e4b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:59.065284 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=852082d3b62f49b48881b81bf3ce1e4b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:59.067271 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:59.128883 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '852082d3b62f49b48881b81bf3ce1e4b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.129570 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '852082d3b62f49b48881b81bf3ce1e4b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.130236 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '852082d3b62f49b48881b81bf3ce1e4b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.142757 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '852082d3b62f49b48881b81bf3ce1e4b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.152511 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '852082d3b62f49b48881b81bf3ce1e4b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.166660 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1440/2898] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:45:58 2022] DELETE /identity/v3/users/852082d3b62f49b48881b81bf3ce1e4b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:45:59.362811 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:59.364803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:59.379259 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10111a47-87c9-4804-aa26-9fc9857e865c admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:59.433148 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:45:59.445590 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:59.446552 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:59.446986 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:59.447282 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] PATH_INFO: `/v3/projects/63e3d5bb531543c4afae2a644b2c6870` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:59.448464 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e3d5bb531543c4afae2a644b2c6870)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:59.450462 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:59.504585 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e3d5bb531543c4afae2a644b2c6870', 'request_id': 'req-bdf22272-119b-45b5-bff6-456465e831db'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.513948 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e3d5bb531543c4afae2a644b2c6870', 'request_id': 'req-bdf22272-119b-45b5-bff6-456465e831db'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.521165 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e3d5bb531543c4afae2a644b2c6870', 'request_id': 'req-bdf22272-119b-45b5-bff6-456465e831db'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.531600 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] The token cache is being invalidate because project 63e3d5bb531543c4afae2a644b2c6870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:45:59.531887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.532523 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.534097 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1459/2899] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:45:59 2022] DELETE /identity/v3/projects/63e3d5bb531543c4afae2a644b2c6870 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:46:09.163426 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:46:09.528803 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:46:58.516048 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:46:58.517708 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:46:58.591452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862f716-a674-4075-933c-63757504ab26 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:46:58.651519 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964bb62-b999-4560-959c-7b4727b3f561 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:46:58.652182 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964bb62-b999-4560-959c-7b4727b3f561 service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:46:58.652452 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964bb62-b999-4560-959c-7b4727b3f561 service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:46:58.652741 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964bb62-b999-4560-959c-7b4727b3f561 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:46:58.654538 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a964bb62-b999-4560-959c-7b4727b3f561 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:46:58.665848 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964bb62-b999-4560-959c-7b4727b3f561 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:46:58.667455 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964bb62-b999-4560-959c-7b4727b3f561 service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:46:58.677987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1441/2900] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:46:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:47:08.673528 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=58 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:47:27.815478 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:47:27.817066 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:47:27.832669 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf22272-119b-45b5-bff6-456465e831db admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:47:27.837786 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8406ba-0d0c-49dd-bcc6-c92f07e49e3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:47:27.838406 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8406ba-0d0c-49dd-bcc6-c92f07e49e3d service placement] REQUEST_METHOD: `GET` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:47:27.838700 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8406ba-0d0c-49dd-bcc6-c92f07e49e3d service placement] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:47:27.838953 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8406ba-0d0c-49dd-bcc6-c92f07e49e3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:47:27.840758 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-5b8406ba-0d0c-49dd-bcc6-c92f07e49e3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:47:27.853360 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8406ba-0d0c-49dd-bcc6-c92f07e49e3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:47:27.855851 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8406ba-0d0c-49dd-bcc6-c92f07e49e3d service placement] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:47:27.866140 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1460/2901] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:47:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:47:30.872781 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964bb62-b999-4560-959c-7b4727b3f561 service placement] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:47:30.874983 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-a964bb62-b999-4560-959c-7b4727b3f561 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:47:30.953594 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964bb62-b999-4560-959c-7b4727b3f561 service placement] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:47:30.959343 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:47:30.959986 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:47:30.960262 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:47:30.960533 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] PATH_INFO: `/v3/users/057e6fd8802348b988baaa181ee163da` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:47:30.961776 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=057e6fd8802348b988baaa181ee163da)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:47:30.963791 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:47:31.021476 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '057e6fd8802348b988baaa181ee163da'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.022127 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '057e6fd8802348b988baaa181ee163da'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.022724 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '057e6fd8802348b988baaa181ee163da'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.034438 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '057e6fd8802348b988baaa181ee163da'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.043038 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '057e6fd8802348b988baaa181ee163da'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.056885 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1442/2902] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:47:30 2022] DELETE /identity/v3/users/057e6fd8802348b988baaa181ee163da => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:47:31.246225 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8406ba-0d0c-49dd-bcc6-c92f07e49e3d service placement] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:47:31.261215 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8406ba-0d0c-49dd-bcc6-c92f07e49e3d service placement] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:47:31.326269 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:47:31.326806 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:47:31.327085 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:47:31.327359 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] PATH_INFO: `/v3/projects/176f865ce7f542f4a7af595b17e2324a` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:47:31.328596 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=176f865ce7f542f4a7af595b17e2324a)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:47:31.330612 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:47:31.380529 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '176f865ce7f542f4a7af595b17e2324a', 'request_id': 'req-4dbbaa14-c042-44a1-ad64-d986f6075b95'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.390045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '176f865ce7f542f4a7af595b17e2324a', 'request_id': 'req-4dbbaa14-c042-44a1-ad64-d986f6075b95'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.396852 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '176f865ce7f542f4a7af595b17e2324a', 'request_id': 'req-4dbbaa14-c042-44a1-ad64-d986f6075b95'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.407724 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] The token cache is being invalidate because project 176f865ce7f542f4a7af595b17e2324a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:47:31.408183 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.408927 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.410543 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1461/2903] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:47:31 2022] DELETE /identity/v3/projects/176f865ce7f542f4a7af595b17e2324a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:47:31.416404 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:47:31.418674 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:47:31.439987 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655bfea5-bb35-4994-9b0c-6541e17fdfd3 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:47:31.499028 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:47:31.499686 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:47:31.499959 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:47:31.500225 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] PATH_INFO: `/v3/users/7142457ad40f4958b2449e380f738a7b` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:47:31.501419 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7142457ad40f4958b2449e380f738a7b)` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:47:31.503503 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:47:31.525710 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:47:31.555181 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7142457ad40f4958b2449e380f738a7b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.555823 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7142457ad40f4958b2449e380f738a7b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.556414 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7142457ad40f4958b2449e380f738a7b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.568282 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7142457ad40f4958b2449e380f738a7b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.578743 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.notifications [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7142457ad40f4958b2449e380f738a7b'} {{(pid=81634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.591601 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1443/2904] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 12:47:31 2022] DELETE /identity/v3/users/7142457ad40f4958b2449e380f738a7b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:47:31.777782 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] Authenticating user token {{(pid=81635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:47:31.779439 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.fernet_utils [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:47:31.794079 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbaa14-c042-44a1-ad64-d986f6075b95 admin admin] Validating token access rules against request {{(pid=81635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:47:31.854703 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46065e-0845-4696-a674-dcaa86231df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe15d570e61b4d028bf4767532315b26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '66ac24841ce6483cbf8285ea30e957f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:47:31.855422 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46065e-0845-4696-a674-dcaa86231df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:47:31.855614 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46065e-0845-4696-a674-dcaa86231df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:47:31.855887 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46065e-0845-4696-a674-dcaa86231df4 admin admin] PATH_INFO: `/v3/projects/cce178327b0a4767b777f4b9102d4ff5` {{(pid=81635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:47:31.857074 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46065e-0845-4696-a674-dcaa86231df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cce178327b0a4767b777f4b9102d4ff5)` {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:47:31.859004 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46065e-0845-4696-a674-dcaa86231df4 admin admin] RBAC: Authorization granted {{(pid=81635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:47:31.882483 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:47:31.913761 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-aa46065e-0845-4696-a674-dcaa86231df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cce178327b0a4767b777f4b9102d4ff5', 'request_id': 'req-aa46065e-0845-4696-a674-dcaa86231df4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.924045 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-aa46065e-0845-4696-a674-dcaa86231df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cce178327b0a4767b777f4b9102d4ff5', 'request_id': 'req-aa46065e-0845-4696-a674-dcaa86231df4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.931219 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-aa46065e-0845-4696-a674-dcaa86231df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cce178327b0a4767b777f4b9102d4ff5', 'request_id': 'req-aa46065e-0845-4696-a674-dcaa86231df4'} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.942702 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-aa46065e-0845-4696-a674-dcaa86231df4 admin admin] The token cache is being invalidate because project cce178327b0a4767b777f4b9102d4ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:47:31.942991 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-aa46065e-0845-4696-a674-dcaa86231df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.943749 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG keystone.notifications [None req-aa46065e-0845-4696-a674-dcaa86231df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:47:31.945395 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: [pid: 81635|app: 0|req: 1462/2905] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 12:47:31 2022] DELETE /identity/v3/projects/cce178327b0a4767b777f4b9102d4ff5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:47:41.588372 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:47:41.939537 dv-cinder-846054-1-iscsi devstack@keystone.service[81635]: DEBUG dbcounter [-] [81635] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:52:01.487580 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] Authenticating user token {{(pid=81634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:52:01.489365 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:52:01.655979 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57985152-bce8-44d8-95af-ecff26607376 admin admin] Validating token access rules against request {{(pid=81634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:52:01.715667 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f73563-1bff-4e0d-af35-df0e63c8888a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db648aaa06b476e9a3cafbab832ee68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70a5f9cf0499405d82f5f5a47c2877eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:52:01.716267 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f73563-1bff-4e0d-af35-df0e63c8888a service placement] REQUEST_METHOD: `GET` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:52:01.716537 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f73563-1bff-4e0d-af35-df0e63c8888a service placement] SCRIPT_NAME: `/identity` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:52:01.716834 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f73563-1bff-4e0d-af35-df0e63c8888a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:52:01.718612 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.fernet_utils [None req-00f73563-1bff-4e0d-af35-df0e63c8888a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:52:01.730157 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f73563-1bff-4e0d-af35-df0e63c8888a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:52:01.731790 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f73563-1bff-4e0d-af35-df0e63c8888a service placement] RBAC: Authorization granted {{(pid=81634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:52:01.741512 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: [pid: 81634|app: 0|req: 1444/2906] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 12:52:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:52:11.737448 dv-cinder-846054-1-iscsi devstack@keystone.service[81634]: DEBUG dbcounter [-] [81634] Writing DB stats keystone:SELECT=89 {{(pid=81634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}